-- Logs begin at Wed 2022-08-24 10:36:58 UTC, end at Wed 2022-08-24 11:58:21 UTC. -- Aug 24 11:10:28.960136 dv-cinder-854274-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 11:10:28.969584 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 11:10:28.971818 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 11:10:28 2022] *** Aug 24 11:10:28.971818 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 11:10:28.971818 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 11:10:28.971818 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: nodename: dv-cinder-854274-2-iscsi Aug 24 11:10:28.971818 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: machine: x86_64 Aug 24 11:10:28.971818 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: clock source: unix Aug 24 11:10:28.971818 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: pcre jit disabled Aug 24 11:10:28.972216 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: detected number of CPU cores: 4 Aug 24 11:10:28.972216 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: current working directory: / Aug 24 11:10:28.972216 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: detected binary path: /usr/bin/uwsgi-core Aug 24 11:10:28.972216 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: your processes number limit is 47680 Aug 24 11:10:28.972216 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: your memory page size is 4096 bytes Aug 24 11:10:28.972216 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: detected max file descriptor number: 2048 Aug 24 11:10:28.972216 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: lock engine: pthread robust mutexes Aug 24 11:10:28.972216 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: thunder lock: enabled Aug 24 11:10:28.972590 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 11:10:28.972590 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 11:10:29.016895 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: Python main interpreter initialized at 0x556bd10c39c0 Aug 24 11:10:29.016895 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: python threads support enabled Aug 24 11:10:29.016895 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: your server socket listen backlog is limited to 100 connections Aug 24 11:10:29.016895 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: your mercy for graceful operations on workers is 90 seconds Aug 24 11:10:29.017271 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 11:10:29.017345 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: *** Operational MODE: preforking *** Aug 24 11:10:29.017591 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: *** uWSGI is running in multiple interpreter mode *** Aug 24 11:10:29.017591 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: spawned uWSGI master process (pid: 81694) Aug 24 11:10:29.017800 dv-cinder-854274-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 11:10:29.018675 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: spawned uWSGI worker 1 (pid: 81706, cores: 1) Aug 24 11:10:29.018766 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: spawned uWSGI worker 2 (pid: 81707, cores: 1) Aug 24 11:10:29.018914 dv-cinder-854274-2-iscsi devstack@keystone.service[81694]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 11:10:33.343721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 11:10:33.343721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: from cryptography.utils import int_from_bytes Aug 24 11:10:33.344521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 11:10:33.344521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: from cryptography.utils import int_from_bytes Aug 24 11:10:33.347639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 11:10:33.347639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: from cryptography.utils import int_from_bytes Aug 24 11:10:33.348096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 11:10:33.348096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: from cryptography.utils import int_from_bytes Aug 24 11:10:33.394146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 11:10:33.394344 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 11:10:33.394509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] command line args: [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 11:10:33.394772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 11:10:33.394944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ================================================================================ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 11:10:33.395192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] admin_token = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.395434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] config_dir = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.395617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.395792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] config_source = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.395965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] debug = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.396157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.396665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.396759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.397002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.397248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.397473 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.397648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_config_append = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.397821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.398047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_dir = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.398168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_file = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.398388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.398629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.398884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.399113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.399287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.399459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.399696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.399935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.400146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 11:10:33.400255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.400354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 11:10:33.400471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.400558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] command line args: [] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 11:10:33.400636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.400821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 11:10:33.400921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.400997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ================================================================================ {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 11:10:33.401056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.401248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] admin_token = **** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.401348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.401496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] config_dir = [] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.402112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.402241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.402593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.402695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] config_source = [] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.402764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] publish_errors = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.402924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] debug = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.403042 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.403195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.403321 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.403396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.403471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.403597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.403701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.403827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.403932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.404055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] standard_threads = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.404145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.404282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.404364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] list_limit = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.404472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] log_config_append = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.404555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.404653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.404730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.404884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] log_dir = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_journal = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_json = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] log_file = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_stderr = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] use_syslog = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.405826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.405995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.406073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.406176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.406277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.406399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.406474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.406585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.406701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.406831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.406934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.407069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.407176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.407304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.407405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.407539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.407647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.407772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.407888 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.408018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.408134 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.408264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.408367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.408540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.408625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.408842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.409077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.84/identity {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.409166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.409244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] publish_errors = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.409320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.409484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.409638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.409711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.409780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.409999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.410098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.410179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.410251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.410403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.410483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.410638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] standard_threads = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.410721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.410884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.410966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.411115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.411191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.411291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.411375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.411456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] use_journal = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.411607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.411690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] use_json = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.411832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.411914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] use_stderr = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.412005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] use_syslog = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.412088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.412238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 11:10:33.412314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.412496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.412580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.412735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.413413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.413413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.413605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.413683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.413759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.413829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.413905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.413975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414042 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.414939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.415014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.415165 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.415242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.415389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.415465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.415616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.415694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.415848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.415930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.416090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.416166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.416319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.416396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.416550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.416631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.416775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.416871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.417027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.417105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.418970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.419157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.419225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.419393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.419464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.419616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.419698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.419849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.419966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.420095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.420203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.420308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.420405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.420509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.420594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.420762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.420872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.421017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.421109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.421243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.421326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.421474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.421559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.421704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.421782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.421968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.422107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.422185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.422301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.422431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.422509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.422670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.422758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.422901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.422988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.423130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.423206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.423358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.423446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.423587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.423666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.423819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.423898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.424050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.424139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.424285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.424364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.424514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.424596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.424742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.424891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.425015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.425132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.425232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.425310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.425465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.425554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.425743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.425827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.425943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.426021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.426178 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.426264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.426411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.426489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.426647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.426725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.426878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.426956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.427108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.427191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.427337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.427413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.427560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.427633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.427778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.427856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.428003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.428080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.428235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.428329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.428464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.428571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.428701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.428788 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.428953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.429033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.429197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.429276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.429429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.429508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.429681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.429806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.429931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.430010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.430167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.430243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.430393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.external = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.430470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.430668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.430751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.430907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.430984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.431155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.431232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.431395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.431482 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.password = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.431629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] auth.token = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.431705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.431865 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.431942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.432097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.432183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.432333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.432410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.432558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.432634 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.432872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.432984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.433108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.433311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.433390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.433554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.433644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.433797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.433881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.434041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.434120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.434281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.434366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.434511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.434589 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.434748 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.434897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.435985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.436139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.436224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.436372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.436450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.436674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.436859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.436968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.437048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.437157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.437235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.437397 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.437488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.437656 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.437736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.437912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.437995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.438152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.438242 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.438376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.438454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] auth.external = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.438607 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.438688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.438834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.438919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.439081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.439160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.439312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.439390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] auth.password = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.439560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.439640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] auth.token = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.439792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.439870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.440027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.440104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.440255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.440333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.440493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.440570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.440747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.440955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] credential.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.441992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.442155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.442233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.442399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.442479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.442625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.442703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.442850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.442928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.443075 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.443154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.443313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.443390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.443553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.443630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.443793 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.443872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.444043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.444121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.444282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.444360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.444547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.444666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.444783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.444960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.445081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.445157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.445236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.445309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] federation.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.445468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.445547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.445692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.445770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.445921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.446001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.446154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.446234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.446386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.446464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.446629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.446708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.446853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.446948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.447092 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.447171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.447332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.447418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.447561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.447640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.447799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.447885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.448039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.448118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.448261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.448339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.448622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.448728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.448853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.448930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.449082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.449160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.449330 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.449422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.449564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.449764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.449842 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.450005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.450083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.450245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.450326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.450495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.450574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.450741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.450819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.450970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.451049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.451196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.451275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.451441 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.451519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.451685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.451765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.451917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.451995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.452149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.452227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.452379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.452458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.452604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.452682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.452860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.452942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.453117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.453242 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.453343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.453493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.453572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.453728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.453814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.454005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.454083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.454250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.454329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.454473 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.454559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.454704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.454783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.454938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.455015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.455168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.455254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.455394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.455474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.455634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.455712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.455875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.455960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.456113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.456182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.456342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.456418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.456579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.456664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.456838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.456920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.457065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.457142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.457298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.457387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.457531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.457609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.457761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.457839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.457992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.458078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.458232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.458310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.458474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.458551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.458703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.458792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.458941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.459020 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.459164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.459241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.459402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.459488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.459789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.459898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.459966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.460028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.460138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.460228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.460366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.460445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.460603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.460680 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.460853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.460947 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.461088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.461211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.461346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.461423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.461595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.461675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.461835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.461912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.462066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.462144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.462294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.462372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.462529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.462607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.462758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.462836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.462999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.463112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.463231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.463351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.463463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.463604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.463758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] role.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.463856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.463935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] role.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.464688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.464777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.464977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.470626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.470783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.470884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.470958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471030 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.471977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.472964 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] resource.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.473943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.474924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] role.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] role.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475425 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.475984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476053 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.476945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477164 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477501 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477799 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.477940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.478948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.479716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.480473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.480625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.480708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.480892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.480975 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.481137 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.481303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] token.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.481396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.481472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.481587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.481666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.481817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.481895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.481975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81706) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 11:10:33.482102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.482396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.482634 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.482880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.483124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.483372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.483609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.483851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.484087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.484333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.484582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.484822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.485073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.485369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.485632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.connection = **** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.485880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.486130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.486369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.486612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.486848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.487086 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.487322 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.487560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.487806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.488044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.488355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.488598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.488869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.489110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.489356 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.489588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.489828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.490064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 11:10:33.490233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81707) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 11:10:33.510791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.511646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.511837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.515451 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.515687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.515914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.520172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.524431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.548572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.548822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.556813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.557105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.610078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.610638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.661929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.662164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.662417 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.662655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.835930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.836169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.836376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.836590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.836820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.837053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.844712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.844972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.845198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.845419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.845638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.846716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.866279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.866393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.866479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.866580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.866663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.866809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.866891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.867037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.867128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.867258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.867340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.867467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.867547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.867677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.879184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.879587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.897721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.897944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.898047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.898166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.898249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.898385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.898466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.898600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.898690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.898811 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.898893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.899021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81706) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.899102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.899276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81707) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 11:10:33.916147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.916353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 11:10:33.916567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.916693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.916849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 11:10:33.917040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 11:10:33.917200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.917320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.917936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 11:10:33.918219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.918312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.918476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.918627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.918890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.918970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.919233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.928302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.928577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.928763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.928890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.929070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.930501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.930587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.930587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.936214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.936758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.939164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.939447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.939728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.939809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.940056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.940151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.940362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.940454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.940621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.940703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.940917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.941014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.941191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.941272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.941462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.941573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.941740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.941849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.942119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.942198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.942498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.942779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.965568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.966426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.967093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.967375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.967896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.968187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.972096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.972374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.972656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.973010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.973298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.973583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.978998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.979278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.980620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.982502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.984253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.984541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.984840 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.985170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:33.985259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.985432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.985516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.985709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.985791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.985977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.986067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.986243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.986323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.986530 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.986804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:33.987077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.003744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.004028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.004715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.005028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.007904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.009064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.011237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.011527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.012369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.012677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.015756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.017013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.018904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.020222 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.022142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.022682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.022810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.023000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.023293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.023560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.023650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.023878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.024164 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.024449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.024735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.025050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.040018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.040314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.040579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.040862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.042032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.042971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.043449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.043722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.048355 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.049900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.050176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.050745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.052318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.052598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.052785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.053087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.053375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.053661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.053945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.055365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.057129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.057129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.057129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.057129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.065411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.065691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.065967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.066244 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.066521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.066794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.067063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.068121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.068448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.068736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.069054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.069336 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.069633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.069921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.082792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.085711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.085850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.088734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.088870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.091003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.092426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.092508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.092757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.093068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.094004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.096378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.096808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.097272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.099526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.101053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.103946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.104252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.104518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.108618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.108708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.109357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.112350 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.112642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.112983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.113273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.113538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.113826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.114104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.122381 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81706) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.122604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.123075 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.123519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.123813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.124099 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.124376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.124598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.124692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.124938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.125157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.125241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.125493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.125653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.125760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.126040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.126307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81706) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.135882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81707) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 11:10:34.135882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.135882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.135882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.137469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.137469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.137469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.137469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.137908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.138420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.138933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.139300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.139652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.139922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81707) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 11:10:34.162071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556bd10c39c0 pid: 81706 (default app) Aug 24 11:10:34.165560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2308b3-d2d0-4a6c-81ad-39f5e75ead85 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:34.165844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2308b3-d2d0-4a6c-81ad-39f5e75ead85 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:34.166120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2308b3-d2d0-4a6c-81ad-39f5e75ead85 None None] PATH_INFO: `/v3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:34.170525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1/1] 10.222.0.84 () {50 vars in 791 bytes} [Wed Aug 24 11:10:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:34.179870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556bd10c39c0 pid: 81707 (default app) Aug 24 11:10:41.732751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33737c68-d208-4adc-ac76-3bec3968016d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:41.733275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33737c68-d208-4adc-ac76-3bec3968016d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:41.733358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33737c68-d208-4adc-ac76-3bec3968016d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:41.734624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 2/2] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:10:41.743473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:41.743872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:41.744024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:41.815995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: INFO dbcounter [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] Registered counter for database keystone Aug 24 11:10:41.836976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG oslo_db.sqlalchemy.engines [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81707) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 11:10:41.843390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writer thread running {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 11:10:42.052881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG passlib.handlers.bcrypt [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81707) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 11:10:42.053225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG passlib.handlers.bcrypt [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81707) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 11:10:42.108244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:42.109897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:42.238347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1/3] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 11:10:42.677439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33737c68-d208-4adc-ac76-3bec3968016d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:42.746310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: INFO dbcounter [None req-33737c68-d208-4adc-ac76-3bec3968016d None None] Registered counter for database keystone Aug 24 11:10:42.766267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG oslo_db.sqlalchemy.engines [None req-33737c68-d208-4adc-ac76-3bec3968016d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81706) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 11:10:42.772710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writer thread running {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 11:10:42.893204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33737c68-d208-4adc-ac76-3bec3968016d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:42.897493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35bea5-57be-417a-ac16-4c04aa03202e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:42.898120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35bea5-57be-417a-ac16-4c04aa03202e admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:42.898397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35bea5-57be-417a-ac16-4c04aa03202e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:42.898665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35bea5-57be-417a-ac16-4c04aa03202e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:42.907008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35bea5-57be-417a-ac16-4c04aa03202e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:42.958777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING keystone.common.rbac_enforcer.enforcer [None req-db35bea5-57be-417a-ac16-4c04aa03202e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 11:10:42.959118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35bea5-57be-417a-ac16-4c04aa03202e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:42.969089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 3/4] 10.222.0.84 () {58 vars in 1324 bytes} [Wed Aug 24 11:10:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:10:42.974987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:42.989783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff8df1-f4c8-4924-b718-4a2d490956f9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:42.994214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90b57a-ee84-45a3-84ee-fb8231e4ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:42.995922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90b57a-ee84-45a3-84ee-fb8231e4ca23 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:42.996203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90b57a-ee84-45a3-84ee-fb8231e4ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:42.996468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90b57a-ee84-45a3-84ee-fb8231e4ca23 admin admin] PATH_INFO: `/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:42.997998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90b57a-ee84-45a3-84ee-fb8231e4ca23 admin admin] RBAC: Authorizing `identity:get_project(project_id=cd4c9f105cf4428d9f0cb67e8376cb7e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:43.050650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1d90b57a-ee84-45a3-84ee-fb8231e4ca23 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 11:10:43.050990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90b57a-ee84-45a3-84ee-fb8231e4ca23 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:43.055516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 2/5] 10.222.0.84 () {56 vars in 1207 bytes} [Wed Aug 24 11:10:42 2022] GET /identity/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:44.948761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aef143-82ce-4516-9098-32fd25489611 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:44.949275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aef143-82ce-4516-9098-32fd25489611 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:44.949275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aef143-82ce-4516-9098-32fd25489611 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:44.953304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 4/6] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:10:44.958676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7afa1-fe95-4891-aca2-5798a85f1073 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:44.959100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7afa1-fe95-4891-aca2-5798a85f1073 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:44.959213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7afa1-fe95-4891-aca2-5798a85f1073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:44.978513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-39d7afa1-fe95-4891-aca2-5798a85f1073 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:44.980858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-39d7afa1-fe95-4891-aca2-5798a85f1073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:44.988490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 3/7] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 11:10:45.261151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aef143-82ce-4516-9098-32fd25489611 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:45.273022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aef143-82ce-4516-9098-32fd25489611 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:45.277718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05576385-65cf-4a00-95de-26b87aa545aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:45.278297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576385-65cf-4a00-95de-26b87aa545aa admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:45.278569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576385-65cf-4a00-95de-26b87aa545aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:45.278835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576385-65cf-4a00-95de-26b87aa545aa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:45.286356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05576385-65cf-4a00-95de-26b87aa545aa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:45.288113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05576385-65cf-4a00-95de-26b87aa545aa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:45.297469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 5/8] 10.222.0.84 () {58 vars in 1324 bytes} [Wed Aug 24 11:10:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:10:45.302813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7afa1-fe95-4891-aca2-5798a85f1073 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:45.312106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7afa1-fe95-4891-aca2-5798a85f1073 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:45.316374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e76d7-ac3c-4022-b59a-911a3297e9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:45.317028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e76d7-ac3c-4022-b59a-911a3297e9dd admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:45.317401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e76d7-ac3c-4022-b59a-911a3297e9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:45.317673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e76d7-ac3c-4022-b59a-911a3297e9dd admin admin] PATH_INFO: `/v3/users/4fac3b46e24449e193279323e9d64fe0` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:45.318965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e76d7-ac3c-4022-b59a-911a3297e9dd admin admin] RBAC: Authorizing `identity:get_user(user_id=4fac3b46e24449e193279323e9d64fe0)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:45.320775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e76d7-ac3c-4022-b59a-911a3297e9dd admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:45.322309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 4/9] 10.222.0.84 () {56 vars in 1198 bytes} [Wed Aug 24 11:10:45 2022] GET /identity/v3/users/4fac3b46e24449e193279323e9d64fe0 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:47.300485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ed8cc-96a0-4058-9085-6ebd8f1e3329 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:47.301101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ed8cc-96a0-4058-9085-6ebd8f1e3329 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:47.301101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ed8cc-96a0-4058-9085-6ebd8f1e3329 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:47.302356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 6/10] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:47.308129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bae5f6-665b-423b-8334-476fccce0b8f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:47.308558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bae5f6-665b-423b-8334-476fccce0b8f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:47.308937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bae5f6-665b-423b-8334-476fccce0b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:47.326754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-09bae5f6-665b-423b-8334-476fccce0b8f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:47.327924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-09bae5f6-665b-423b-8334-476fccce0b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:47.333944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d794047-d030-4e09-a839-8ef90dce71c4 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:47.334227 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d794047-d030-4e09-a839-8ef90dce71c4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:47.334496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d794047-d030-4e09-a839-8ef90dce71c4 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:47.335841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 7/11] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:10:47.341073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45243d-126f-415e-b1bc-de9b2215fa71 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:47.341352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45243d-126f-415e-b1bc-de9b2215fa71 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:47.341623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45243d-126f-415e-b1bc-de9b2215fa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:47.383922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG passlib.handlers.bcrypt [None req-9c45243d-126f-415e-b1bc-de9b2215fa71 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81706) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 11:10:47.384339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG passlib.handlers.bcrypt [None req-9c45243d-126f-415e-b1bc-de9b2215fa71 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81706) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 11:10:47.408650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 5/12] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:10:47.430552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9c45243d-126f-415e-b1bc-de9b2215fa71 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:47.432120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9c45243d-126f-415e-b1bc-de9b2215fa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:47.536601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 8/13] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:10:47.684127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bae5f6-665b-423b-8334-476fccce0b8f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:47.707606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:47.792205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bae5f6-665b-423b-8334-476fccce0b8f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:47.800727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45243d-126f-415e-b1bc-de9b2215fa71 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:47.825598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb6900-a34a-4ec3-8f71-8ef8c854dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:47.826233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb6900-a34a-4ec3-8f71-8ef8c854dd45 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:47.826511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb6900-a34a-4ec3-8f71-8ef8c854dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:47.826782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb6900-a34a-4ec3-8f71-8ef8c854dd45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:47.838136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb6900-a34a-4ec3-8f71-8ef8c854dd45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-77fb6900-a34a-4ec3-8f71-8ef8c854dd45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:10:47.847180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:10:47.848280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:10:47.849127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 6/14] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:10:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:10:47.854185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb6900-a34a-4ec3-8f71-8ef8c854dd45 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:47.904278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45243d-126f-415e-b1bc-de9b2215fa71 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:47.906515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:47.935115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:47.941376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61852d77-222c-4c79-8ec1-cee78c934d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:47.942220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61852d77-222c-4c79-8ec1-cee78c934d5e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:47.942496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61852d77-222c-4c79-8ec1-cee78c934d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:47.942759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61852d77-222c-4c79-8ec1-cee78c934d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:47.952972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb6900-a34a-4ec3-8f71-8ef8c854dd45 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:47.986783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefb000-6756-4592-957d-1fc68fdd9e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:47.988033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefb000-6756-4592-957d-1fc68fdd9e66 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:47.988311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefb000-6756-4592-957d-1fc68fdd9e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:47.988578 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefb000-6756-4592-957d-1fc68fdd9e66 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:47.989252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefb000-6756-4592-957d-1fc68fdd9e66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:10:47.989549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefb000-6756-4592-957d-1fc68fdd9e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:47.990925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefb000-6756-4592-957d-1fc68fdd9e66 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:48.002764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 7/15] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:10:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:10:48.008689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefb000-6756-4592-957d-1fc68fdd9e66 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:48.012700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61852d77-222c-4c79-8ec1-cee78c934d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:48.014263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61852d77-222c-4c79-8ec1-cee78c934d5e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:48.106734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefb000-6756-4592-957d-1fc68fdd9e66 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:48.110767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 9/16] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:10:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:10:48.116322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61852d77-222c-4c79-8ec1-cee78c934d5e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:48.119008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:48.140402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163eb09-8c34-403b-890d-ce6c2a6aca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:48.141059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163eb09-8c34-403b-890d-ce6c2a6aca4b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:48.141344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163eb09-8c34-403b-890d-ce6c2a6aca4b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:48.141611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163eb09-8c34-403b-890d-ce6c2a6aca4b None admin] PATH_INFO: `/v3/users/4fac3b46e24449e193279323e9d64fe0` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:48.142739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163eb09-8c34-403b-890d-ce6c2a6aca4b None admin] RBAC: Authorizing `identity:get_user(user_id=4fac3b46e24449e193279323e9d64fe0)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:48.144182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163eb09-8c34-403b-890d-ce6c2a6aca4b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:48.145771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 8/17] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:10:48 2022] GET /identity/v3/users/4fac3b46e24449e193279323e9d64fe0 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:10:48.151679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163eb09-8c34-403b-890d-ce6c2a6aca4b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:48.165795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:48.216003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61852d77-222c-4c79-8ec1-cee78c934d5e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:48.249743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163eb09-8c34-403b-890d-ce6c2a6aca4b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:48.250397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38008e4e-746d-4e98-a622-1fad139c8885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:48.251239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38008e4e-746d-4e98-a622-1fad139c8885 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:48.251518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38008e4e-746d-4e98-a622-1fad139c8885 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:48.251787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38008e4e-746d-4e98-a622-1fad139c8885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:48.270034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38008e4e-746d-4e98-a622-1fad139c8885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:48.271499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38008e4e-746d-4e98-a622-1fad139c8885 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:48.284240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53239c-25f6-434c-aabb-6226c63a18d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:48.284896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53239c-25f6-434c-aabb-6226c63a18d9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:48.285176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53239c-25f6-434c-aabb-6226c63a18d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:48.285445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53239c-25f6-434c-aabb-6226c63a18d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:48.286547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53239c-25f6-434c-aabb-6226c63a18d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:48.287975 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53239c-25f6-434c-aabb-6226c63a18d9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:48.289548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 9/18] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:10:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-38008e4e-746d-4e98-a622-1fad139c8885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:10:48.289989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:10:48.291001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:10:48.291824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:10:48.291824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:10:48.291824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:10:48.291824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 10/19] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:10:48.295745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53239c-25f6-434c-aabb-6226c63a18d9 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:48.296575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38008e4e-746d-4e98-a622-1fad139c8885 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:48.352050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:48.375941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:48.395337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53239c-25f6-434c-aabb-6226c63a18d9 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:48.397753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38008e4e-746d-4e98-a622-1fad139c8885 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:48.429868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9ad7d-1103-441a-bbdd-67353796d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:48.430389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9ad7d-1103-441a-bbdd-67353796d67d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:48.430666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9ad7d-1103-441a-bbdd-67353796d67d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:48.430938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9ad7d-1103-441a-bbdd-67353796d67d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:48.431367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0edc44-0584-4742-b7e4-3d947ea6b2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:48.431647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9ad7d-1103-441a-bbdd-67353796d67d None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.domain.id=default, user.id=4fac3b46e24449e193279323e9d64fe0) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:10:48.431950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9ad7d-1103-441a-bbdd-67353796d67d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:48.432041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0edc44-0584-4742-b7e4-3d947ea6b2fa None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:48.432239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0edc44-0584-4742-b7e4-3d947ea6b2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:48.433141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0edc44-0584-4742-b7e4-3d947ea6b2fa None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:48.433779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0edc44-0584-4742-b7e4-3d947ea6b2fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:10:48.433949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9ad7d-1103-441a-bbdd-67353796d67d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:48.434079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0edc44-0584-4742-b7e4-3d947ea6b2fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:48.435451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0edc44-0584-4742-b7e4-3d947ea6b2fa None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:48.440616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 10/20] 10.222.0.84 () {56 vars in 1322 bytes} [Wed Aug 24 11:10:48 2022] GET /identity/v3/role_assignments?user.id=4fac3b46e24449e193279323e9d64fe0&scope.domain.id=default&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:10:48.456310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 11/21] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:10:50.503706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9d3a4-86f7-4552-b7f2-d1ba7c3c4388 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:50.507990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9d3a4-86f7-4552-b7f2-d1ba7c3c4388 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:50.508289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9d3a4-86f7-4552-b7f2-d1ba7c3c4388 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:50.509764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 11/22] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:50.522213 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cafafc-3fae-4c65-bce3-ea4fadcb125a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:50.522729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cafafc-3fae-4c65-bce3-ea4fadcb125a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:50.523056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cafafc-3fae-4c65-bce3-ea4fadcb125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:50.550293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b4cafafc-3fae-4c65-bce3-ea4fadcb125a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:50.551495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b4cafafc-3fae-4c65-bce3-ea4fadcb125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:50.661209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 12/23] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:10:50.812346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d7303-b7ac-4702-a00f-55d1b6f74e09 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:50.812691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d7303-b7ac-4702-a00f-55d1b6f74e09 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:50.812922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d7303-b7ac-4702-a00f-55d1b6f74e09 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:50.814263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 12/24] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:50.819598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd75d8-374b-4bac-80b6-f17ec81d2b66 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:50.820115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd75d8-374b-4bac-80b6-f17ec81d2b66 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:50.820578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd75d8-374b-4bac-80b6-f17ec81d2b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:50.845772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-08cd75d8-374b-4bac-80b6-f17ec81d2b66 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:50.847302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-08cd75d8-374b-4bac-80b6-f17ec81d2b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:50.900082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:50.930688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 13/25] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:10:51.045228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d7303-b7ac-4702-a00f-55d1b6f74e09 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:51.094266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12be3-35d7-4ec5-9b2b-6da14e13202f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:51.094537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12be3-35d7-4ec5-9b2b-6da14e13202f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:51.094733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12be3-35d7-4ec5-9b2b-6da14e13202f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:51.096079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 14/26] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:51.101040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4790f-1ae5-4381-a02c-a7132013a660 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:51.101322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4790f-1ae5-4381-a02c-a7132013a660 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:51.101597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4790f-1ae5-4381-a02c-a7132013a660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:51.131987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c3c4790f-1ae5-4381-a02c-a7132013a660 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:51.133208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c3c4790f-1ae5-4381-a02c-a7132013a660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:51.242963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d7303-b7ac-4702-a00f-55d1b6f74e09 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:51.306983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 15/27] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:10:51.307889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4790f-1ae5-4381-a02c-a7132013a660 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:51.322040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cda337-aadf-426f-9988-9d4725ed4413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:51.322684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cda337-aadf-426f-9988-9d4725ed4413 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:51.323121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cda337-aadf-426f-9988-9d4725ed4413 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:51.323557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cda337-aadf-426f-9988-9d4725ed4413 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:51.343798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cda337-aadf-426f-9988-9d4725ed4413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-83cda337-aadf-426f-9988-9d4725ed4413 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:10:51.349376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:10:51.350201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:10:51.350930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:10:51.353201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 13/28] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:10:51.355123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47589fb-9e84-4df1-9b0b-e89b42e40810 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:51.355917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47589fb-9e84-4df1-9b0b-e89b42e40810 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:51.356382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47589fb-9e84-4df1-9b0b-e89b42e40810 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:51.359448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 14/29] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:51.361110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47589fb-9e84-4df1-9b0b-e89b42e40810 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:51.373187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:51.485167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4790f-1ae5-4381-a02c-a7132013a660 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:51.506478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:51.533368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95542181-88b9-4f1d-87ca-e94a709a3695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:51.535069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95542181-88b9-4f1d-87ca-e94a709a3695 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:51.535502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95542181-88b9-4f1d-87ca-e94a709a3695 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:51.535921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95542181-88b9-4f1d-87ca-e94a709a3695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:51.547951 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95542181-88b9-4f1d-87ca-e94a709a3695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:51.549890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95542181-88b9-4f1d-87ca-e94a709a3695 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-95542181-88b9-4f1d-87ca-e94a709a3695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:10:51.566713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:10:51.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:10:51.568684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:10:51.568684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:10:51.568684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:10:51.570235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 16/30] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:10:51.571652 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e155cb9-edcf-4c6e-b800-bb2998d62320 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:51.572077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e155cb9-edcf-4c6e-b800-bb2998d62320 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:51.572485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e155cb9-edcf-4c6e-b800-bb2998d62320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:51.594241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0e155cb9-edcf-4c6e-b800-bb2998d62320 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:51.595903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0e155cb9-edcf-4c6e-b800-bb2998d62320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:51.605822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47589fb-9e84-4df1-9b0b-e89b42e40810 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:51.663494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d779f-00ac-4fe4-a7b8-74b3eda51ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:51.664338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d779f-00ac-4fe4-a7b8-74b3eda51ced None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:51.666303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d779f-00ac-4fe4-a7b8-74b3eda51ced None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:51.666846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d779f-00ac-4fe4-a7b8-74b3eda51ced None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:51.668064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d779f-00ac-4fe4-a7b8-74b3eda51ced None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:10:51.668577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d779f-00ac-4fe4-a7b8-74b3eda51ced None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:51.670309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d779f-00ac-4fe4-a7b8-74b3eda51ced None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:51.687643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 15/31] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:10:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:51.687643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d779f-00ac-4fe4-a7b8-74b3eda51ced None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:51.709497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 17/32] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:10:51.710621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e155cb9-edcf-4c6e-b800-bb2998d62320 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:51.821870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:51.836685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e155cb9-edcf-4c6e-b800-bb2998d62320 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:51.860423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:51.879393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe3195-ff46-403d-97e4-211015816bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:51.880389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe3195-ff46-403d-97e4-211015816bca None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:51.880900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe3195-ff46-403d-97e4-211015816bca None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:51.881567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe3195-ff46-403d-97e4-211015816bca None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:51.882503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe3195-ff46-403d-97e4-211015816bca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:51.884457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe3195-ff46-403d-97e4-211015816bca None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:51.919583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d779f-00ac-4fe4-a7b8-74b3eda51ced None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:51.962578 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86fab8-ce19-4b72-b99a-41966adf2c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:51.963346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86fab8-ce19-4b72-b99a-41966adf2c0f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:51.964237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86fab8-ce19-4b72-b99a-41966adf2c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:51.964659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86fab8-ce19-4b72-b99a-41966adf2c0f None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:51.965344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86fab8-ce19-4b72-b99a-41966adf2c0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:10:51.965646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86fab8-ce19-4b72-b99a-41966adf2c0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:51.967129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86fab8-ce19-4b72-b99a-41966adf2c0f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:51.990582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 16/33] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:51.991037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86fab8-ce19-4b72-b99a-41966adf2c0f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:52.102013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86fab8-ce19-4b72-b99a-41966adf2c0f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:52.131518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:52.135607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 18/34] 10.222.0.84 () {60 vars in 1131 bytes} [Wed Aug 24 11:10:51 2022] POST /identity/v3/roles => generated 219 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:10:52.137413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe3195-ff46-403d-97e4-211015816bca None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:52.143692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9eea4-c06b-4fa0-8b68-f8d126282d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:52.144893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9eea4-c06b-4fa0-8b68-f8d126282d3d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:52.145344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9eea4-c06b-4fa0-8b68-f8d126282d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:52.145755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9eea4-c06b-4fa0-8b68-f8d126282d3d None admin] PATH_INFO: `/v3/users/4fac3b46e24449e193279323e9d64fe0` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:52.150510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9eea4-c06b-4fa0-8b68-f8d126282d3d None admin] RBAC: Authorizing `identity:get_user(user_id=4fac3b46e24449e193279323e9d64fe0)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:52.150510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9eea4-c06b-4fa0-8b68-f8d126282d3d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:52.152617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 17/35] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:10:51 2022] GET /identity/v3/users/4fac3b46e24449e193279323e9d64fe0 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:52.153726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9eea4-c06b-4fa0-8b68-f8d126282d3d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:52.344724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe3195-ff46-403d-97e4-211015816bca None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:52.346085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9eea4-c06b-4fa0-8b68-f8d126282d3d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:52.392094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf016026-adb6-4f5c-8dae-c7172b0ac349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:52.393109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf016026-adb6-4f5c-8dae-c7172b0ac349 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:52.393109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf016026-adb6-4f5c-8dae-c7172b0ac349 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:52.393109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf016026-adb6-4f5c-8dae-c7172b0ac349 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:52.393721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf016026-adb6-4f5c-8dae-c7172b0ac349 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:52.395163 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf016026-adb6-4f5c-8dae-c7172b0ac349 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:52.396692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34326609-621c-4de5-a6a1-6b8464f9ea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:52.397343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34326609-621c-4de5-a6a1-6b8464f9ea7d None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:52.397622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34326609-621c-4de5-a6a1-6b8464f9ea7d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:52.399496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34326609-621c-4de5-a6a1-6b8464f9ea7d None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:52.399496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34326609-621c-4de5-a6a1-6b8464f9ea7d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:52.400049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34326609-621c-4de5-a6a1-6b8464f9ea7d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:52.490580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 19/36] 10.222.0.84 () {60 vars in 1131 bytes} [Wed Aug 24 11:10:52 2022] POST /identity/v3/roles => generated 225 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:10:52.491419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34326609-621c-4de5-a6a1-6b8464f9ea7d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:52.503336 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:52.620626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34326609-621c-4de5-a6a1-6b8464f9ea7d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:52.664226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 18/37] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:10:52 2022] POST /identity/v3/projects => generated 301 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:10:52.664226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ead8d-c51c-4ebb-abf6-f7f4e88e57c2 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:52.664226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ead8d-c51c-4ebb-abf6-f7f4e88e57c2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:52.664226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ead8d-c51c-4ebb-abf6-f7f4e88e57c2 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:52.664226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 19/38] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:52.664226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3469775-aa41-4ee3-b990-63315cb6ca4b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:52.664226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3469775-aa41-4ee3-b990-63315cb6ca4b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:52.665146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3469775-aa41-4ee3-b990-63315cb6ca4b None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:52.665146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 20/39] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:52.668823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1b851-d02c-49e8-bc9b-709b77700663 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:52.669125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1b851-d02c-49e8-bc9b-709b77700663 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:52.669402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1b851-d02c-49e8-bc9b-709b77700663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:52.676374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76e358-1a0a-4cce-85a6-0fa3065de53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:52.677814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76e358-1a0a-4cce-85a6-0fa3065de53b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:52.677814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76e358-1a0a-4cce-85a6-0fa3065de53b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:52.677814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76e358-1a0a-4cce-85a6-0fa3065de53b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:52.679164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76e358-1a0a-4cce-85a6-0fa3065de53b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:52.680634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76e358-1a0a-4cce-85a6-0fa3065de53b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:52.688262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 20/40] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:10:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:52.688262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3a78e-ab3d-464d-9b16-13577e0fc435 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:52.688262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3a78e-ab3d-464d-9b16-13577e0fc435 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:52.688262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3a78e-ab3d-464d-9b16-13577e0fc435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:52.697963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-2fd1b851-d02c-49e8-bc9b-709b77700663 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:52.702255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2fd1b851-d02c-49e8-bc9b-709b77700663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:52.707124 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f8e3a78e-ab3d-464d-9b16-13577e0fc435 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:52.708721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f8e3a78e-ab3d-464d-9b16-13577e0fc435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:52.762710 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:52.791582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 21/41] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:10:52.801056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1b851-d02c-49e8-bc9b-709b77700663 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:52.811078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 21/42] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:10:52.912763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1b851-d02c-49e8-bc9b-709b77700663 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:52.959765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd29a1a-9568-4810-ade4-a9c7f94e4dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:52.960447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd29a1a-9568-4810-ade4-a9c7f94e4dad None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:52.960725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd29a1a-9568-4810-ade4-a9c7f94e4dad None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:52.961031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd29a1a-9568-4810-ade4-a9c7f94e4dad None admin] PATH_INFO: `/v3/domains/default/users/4fac3b46e24449e193279323e9d64fe0/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:52.962993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd29a1a-9568-4810-ade4-a9c7f94e4dad None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4fac3b46e24449e193279323e9d64fe0, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:52.965159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd29a1a-9568-4810-ade4-a9c7f94e4dad None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:53.060424 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 22/43] 10.222.0.84 () {58 vars in 1361 bytes} [Wed Aug 24 11:10:52 2022] PUT /identity/v3/domains/default/users/4fac3b46e24449e193279323e9d64fe0/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:10:53.068339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3a78e-ab3d-464d-9b16-13577e0fc435 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:53.079154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:53.094012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd29a1a-9568-4810-ade4-a9c7f94e4dad None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:53.134910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:53.200776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3a78e-ab3d-464d-9b16-13577e0fc435 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:53.241731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448685e2-e434-4cbe-b7df-9aff17ec3665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:53.242370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448685e2-e434-4cbe-b7df-9aff17ec3665 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:53.242646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448685e2-e434-4cbe-b7df-9aff17ec3665 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:53.242917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448685e2-e434-4cbe-b7df-9aff17ec3665 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:53.244018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448685e2-e434-4cbe-b7df-9aff17ec3665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:53.246720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448685e2-e434-4cbe-b7df-9aff17ec3665 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:53.248642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 22/44] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:10:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:53.253925 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448685e2-e434-4cbe-b7df-9aff17ec3665 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:53.335882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd29a1a-9568-4810-ade4-a9c7f94e4dad None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:53.340523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:53.366675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448685e2-e434-4cbe-b7df-9aff17ec3665 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:53.376371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3f6c4-5025-4162-b63b-f551147a1df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:53.377013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3f6c4-5025-4162-b63b-f551147a1df3 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:53.377292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3f6c4-5025-4162-b63b-f551147a1df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:53.377792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3f6c4-5025-4162-b63b-f551147a1df3 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:53.378209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3f6c4-5025-4162-b63b-f551147a1df3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:53.379594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3f6c4-5025-4162-b63b-f551147a1df3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:53.403278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5f91a-9abc-4ab7-8c2b-1d2d78ba3350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:53.403897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5f91a-9abc-4ab7-8c2b-1d2d78ba3350 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:53.404177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5f91a-9abc-4ab7-8c2b-1d2d78ba3350 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:53.404450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5f91a-9abc-4ab7-8c2b-1d2d78ba3350 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:53.405128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5f91a-9abc-4ab7-8c2b-1d2d78ba3350 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:53.406573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5f91a-9abc-4ab7-8c2b-1d2d78ba3350 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:53.456915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 23/45] 10.222.0.84 () {60 vars in 1131 bytes} [Wed Aug 24 11:10:53 2022] POST /identity/v3/roles => generated 223 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:10:53.548827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 23/46] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:10:53 2022] POST /identity/v3/projects => generated 312 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:10:54.982608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c85cf-a73a-435b-a69b-960ca007c32e None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:54.983090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c85cf-a73a-435b-a69b-960ca007c32e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:54.983090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c85cf-a73a-435b-a69b-960ca007c32e None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:54.984420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 24/47] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:54.989871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3324b-a781-4c1c-a8bd-7e619f80a2c2 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:54.990282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3324b-a781-4c1c-a8bd-7e619f80a2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:54.990430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3324b-a781-4c1c-a8bd-7e619f80a2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:55.008109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1cd3324b-a781-4c1c-a8bd-7e619f80a2c2 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:55.009312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1cd3324b-a781-4c1c-a8bd-7e619f80a2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:55.075548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 24/48] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:10:55.369479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c85cf-a73a-435b-a69b-960ca007c32e None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:55.465091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:55.471629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c85cf-a73a-435b-a69b-960ca007c32e None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:55.473802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f2743-8b3d-4279-a440-dc01b39eeefe None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:55.474086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f2743-8b3d-4279-a440-dc01b39eeefe None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:55.474360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f2743-8b3d-4279-a440-dc01b39eeefe None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:55.475707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 25/49] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:55.480998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcf26e-1b75-4c42-97cd-f0e5f0e3fda1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:55.481369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcf26e-1b75-4c42-97cd-f0e5f0e3fda1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:55.481645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcf26e-1b75-4c42-97cd-f0e5f0e3fda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:55.499230 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-45bcf26e-1b75-4c42-97cd-f0e5f0e3fda1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:55.500426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-45bcf26e-1b75-4c42-97cd-f0e5f0e3fda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:55.509028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f6a8b-d8e9-46bb-ac71-5ae36e53dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:55.509608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f6a8b-d8e9-46bb-ac71-5ae36e53dfda None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:55.509887 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f6a8b-d8e9-46bb-ac71-5ae36e53dfda None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:55.510236 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f6a8b-d8e9-46bb-ac71-5ae36e53dfda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:55.520889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f6a8b-d8e9-46bb-ac71-5ae36e53dfda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-234f6a8b-d8e9-46bb-ac71-5ae36e53dfda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:10:55.525835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:10:55.527834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:10:55.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 25/50] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:10:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:10:55.535536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f6a8b-d8e9-46bb-ac71-5ae36e53dfda None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:55.560722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:55.577250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 26/51] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:10:55.642939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f6a8b-d8e9-46bb-ac71-5ae36e53dfda None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:55.676624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eda7e4-beac-4bed-9cd3-64c49b83e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:55.677259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eda7e4-beac-4bed-9cd3-64c49b83e831 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:55.677539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eda7e4-beac-4bed-9cd3-64c49b83e831 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:55.677819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eda7e4-beac-4bed-9cd3-64c49b83e831 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:55.678474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eda7e4-beac-4bed-9cd3-64c49b83e831 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:10:55.678776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eda7e4-beac-4bed-9cd3-64c49b83e831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:55.680162 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eda7e4-beac-4bed-9cd3-64c49b83e831 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:55.689201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 26/52] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:10:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:10:55.694714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcf26e-1b75-4c42-97cd-f0e5f0e3fda1 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:55.791506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcf26e-1b75-4c42-97cd-f0e5f0e3fda1 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:55.824926 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd6c7d-2767-4567-8be6-3b6c1281661f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:55.825553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd6c7d-2767-4567-8be6-3b6c1281661f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:55.825831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd6c7d-2767-4567-8be6-3b6c1281661f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:55.826130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd6c7d-2767-4567-8be6-3b6c1281661f None admin] PATH_INFO: `/v3/users/4fac3b46e24449e193279323e9d64fe0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:55.827433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd6c7d-2767-4567-8be6-3b6c1281661f None admin] RBAC: Authorizing `identity:get_user(user_id=4fac3b46e24449e193279323e9d64fe0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:55.828924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd6c7d-2767-4567-8be6-3b6c1281661f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:55.830697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 27/53] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:10:55 2022] GET /identity/v3/users/4fac3b46e24449e193279323e9d64fe0 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:55.836054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eda7e4-beac-4bed-9cd3-64c49b83e831 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:55.855228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:55.860588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd6c7d-2767-4567-8be6-3b6c1281661f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:55.915515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:55.937695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eda7e4-beac-4bed-9cd3-64c49b83e831 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:55.961043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd6c7d-2767-4567-8be6-3b6c1281661f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:55.971749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bed19-6f61-404d-a850-c5190268d259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:55.972374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bed19-6f61-404d-a850-c5190268d259 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:55.972650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bed19-6f61-404d-a850-c5190268d259 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:55.972957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605bed19-6f61-404d-a850-c5190268d259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:55.974181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bed19-6f61-404d-a850-c5190268d259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:55.976025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605bed19-6f61-404d-a850-c5190268d259 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:55.977641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 27/54] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:10:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:55.983654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bed19-6f61-404d-a850-c5190268d259 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:55.994379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45867300-1220-465c-a42b-8a0fd2e8ddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:55.995531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45867300-1220-465c-a42b-8a0fd2e8ddf0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:55.995881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45867300-1220-465c-a42b-8a0fd2e8ddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:55.996180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45867300-1220-465c-a42b-8a0fd2e8ddf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:55.997317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45867300-1220-465c-a42b-8a0fd2e8ddf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:55.998752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45867300-1220-465c-a42b-8a0fd2e8ddf0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:56.000461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 28/55] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:10:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:10:56.005599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45867300-1220-465c-a42b-8a0fd2e8ddf0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:56.075699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:56.091686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605bed19-6f61-404d-a850-c5190268d259 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:56.128167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ae560-ef1e-4dbc-ac5b-2d804ac5a5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:56.128816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae560-ef1e-4dbc-ac5b-2d804ac5a5df None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:56.129228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae560-ef1e-4dbc-ac5b-2d804ac5a5df None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:56.129517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae560-ef1e-4dbc-ac5b-2d804ac5a5df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:56.130240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ae560-ef1e-4dbc-ac5b-2d804ac5a5df None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.domain.id=default, user.id=4fac3b46e24449e193279323e9d64fe0) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:10:56.130547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ae560-ef1e-4dbc-ac5b-2d804ac5a5df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:56.132009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ae560-ef1e-4dbc-ac5b-2d804ac5a5df None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:56.138212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 28/56] 10.222.0.84 () {56 vars in 1322 bytes} [Wed Aug 24 11:10:55 2022] GET /identity/v3/role_assignments?user.id=4fac3b46e24449e193279323e9d64fe0&scope.domain.id=default&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:10:56.219254 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45867300-1220-465c-a42b-8a0fd2e8ddf0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:56.243499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:56.256107 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ebe43-a73e-4acf-abaa-cb230fc6b177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:56.258077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ebe43-a73e-4acf-abaa-cb230fc6b177 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:56.258077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ebe43-a73e-4acf-abaa-cb230fc6b177 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:56.258077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ebe43-a73e-4acf-abaa-cb230fc6b177 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:56.258077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ebe43-a73e-4acf-abaa-cb230fc6b177 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:56.259424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ebe43-a73e-4acf-abaa-cb230fc6b177 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:56.358156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 29/57] 10.222.0.84 () {60 vars in 1140 bytes} [Wed Aug 24 11:10:56 2022] POST /identity/v3/projects => generated 298 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:10:58.260908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aa12e-2f88-4120-b132-85e9a2bed336 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:58.261428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aa12e-2f88-4120-b132-85e9a2bed336 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:58.261428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aa12e-2f88-4120-b132-85e9a2bed336 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:58.262776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 29/58] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:10:58.267973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b54d8-359c-4c57-93ba-ab233eae29f2 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:58.268385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b54d8-359c-4c57-93ba-ab233eae29f2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:58.268534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b54d8-359c-4c57-93ba-ab233eae29f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:58.287005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c87b54d8-359c-4c57-93ba-ab233eae29f2 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:10:58.288214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c87b54d8-359c-4c57-93ba-ab233eae29f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:10:58.355968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 30/59] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:10:58.626428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8aa12e-2f88-4120-b132-85e9a2bed336 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:58.725832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8aa12e-2f88-4120-b132-85e9a2bed336 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:58.758891 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d42251-4c26-432f-9d3f-0bb1b0026ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:58.759510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d42251-4c26-432f-9d3f-0bb1b0026ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:58.759791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d42251-4c26-432f-9d3f-0bb1b0026ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:58.760064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d42251-4c26-432f-9d3f-0bb1b0026ad5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:58.761752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d42251-4c26-432f-9d3f-0bb1b0026ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:58.763191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d42251-4c26-432f-9d3f-0bb1b0026ad5 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:58.764742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 30/60] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:10:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:10:58.770267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b54d8-359c-4c57-93ba-ab233eae29f2 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:10:58.866119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b54d8-359c-4c57-93ba-ab233eae29f2 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:10:58.868345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:10:58.899847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79551a6-8a15-47d9-a013-1015ca583d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:10:58.900381 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79551a6-8a15-47d9-a013-1015ca583d1b None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:10:58.900655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79551a6-8a15-47d9-a013-1015ca583d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:10:58.901046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79551a6-8a15-47d9-a013-1015ca583d1b None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:10:58.901745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79551a6-8a15-47d9-a013-1015ca583d1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:10:58.905572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79551a6-8a15-47d9-a013-1015ca583d1b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:10:59.003400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 31/61] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:10:58 2022] POST /identity/v3/users => generated 275 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:02.093758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa14c3-ea08-4c34-9212-571ad687ff6e None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.094302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa14c3-ea08-4c34-9212-571ad687ff6e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.094302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa14c3-ea08-4c34-9212-571ad687ff6e None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.095967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 31/62] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:02.100956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce219101-8012-4172-a473-e5760ec5a4a0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.101349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce219101-8012-4172-a473-e5760ec5a4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.101513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce219101-8012-4172-a473-e5760ec5a4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.129867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ce219101-8012-4172-a473-e5760ec5a4a0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:02.131216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ce219101-8012-4172-a473-e5760ec5a4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:02.133221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d5121-0f8c-45c0-bed2-ca645b7d388c None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.133717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d5121-0f8c-45c0-bed2-ca645b7d388c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.134200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d5121-0f8c-45c0-bed2-ca645b7d388c None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.136265 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 32/63] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:02.141648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901462b5-0324-490f-b8ef-39d42fd808c5 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.142175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901462b5-0324-490f-b8ef-39d42fd808c5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.142641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901462b5-0324-490f-b8ef-39d42fd808c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.170327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-901462b5-0324-490f-b8ef-39d42fd808c5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:02.172881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-901462b5-0324-490f-b8ef-39d42fd808c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:02.222854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:02.247661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 32/64] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:02.249451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc70c1c-16bc-4ba6-aa60-aec946f107c4 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.249932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc70c1c-16bc-4ba6-aa60-aec946f107c4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.250421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc70c1c-16bc-4ba6-aa60-aec946f107c4 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.251919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 33/65] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:02.257609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac90c3f-ac2f-4a99-b4b3-32a02a055c45 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.258123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac90c3f-ac2f-4a99-b4b3-32a02a055c45 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.258588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac90c3f-ac2f-4a99-b4b3-32a02a055c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.279221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 33/66] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:02.290494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5ac90c3f-ac2f-4a99-b4b3-32a02a055c45 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:02.291836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5ac90c3f-ac2f-4a99-b4b3-32a02a055c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:02.399525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 34/67] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:02.544308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901462b5-0324-490f-b8ef-39d42fd808c5 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:02.680563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901462b5-0324-490f-b8ef-39d42fd808c5 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:02.696196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac90c3f-ac2f-4a99-b4b3-32a02a055c45 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:02.726025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:02.728751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448d267-24db-468d-aff6-b8866ee56c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:02.729602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448d267-24db-468d-aff6-b8866ee56c27 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.730203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448d267-24db-468d-aff6-b8866ee56c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.730484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448d267-24db-468d-aff6-b8866ee56c27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.743717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448d267-24db-468d-aff6-b8866ee56c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8448d267-24db-468d-aff6-b8866ee56c27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:02.749240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:02.750079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:02.750811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 34/68] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:02.751520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bb893-06d1-4514-af7d-f1063983892b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.751954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bb893-06d1-4514-af7d-f1063983892b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.752359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1bb893-06d1-4514-af7d-f1063983892b None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.753769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 35/69] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:02.754920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009141f-78b8-454b-897f-052b0009e549 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.755341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009141f-78b8-454b-897f-052b0009e549 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.755758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009141f-78b8-454b-897f-052b0009e549 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.757627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 36/70] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:02.759538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009141f-78b8-454b-897f-052b0009e549 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:02.770885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:02.807170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac90c3f-ac2f-4a99-b4b3-32a02a055c45 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:02.840113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a660e7c-2bca-4e38-a248-29b908eef6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:02.840739 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a660e7c-2bca-4e38-a248-29b908eef6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.841044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a660e7c-2bca-4e38-a248-29b908eef6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.841317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a660e7c-2bca-4e38-a248-29b908eef6c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.852377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a660e7c-2bca-4e38-a248-29b908eef6c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-1a660e7c-2bca-4e38-a248-29b908eef6c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:02.860636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:02.861545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:02.862368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 35/71] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:11:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:02.863011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf257-4a6a-4c6e-bd11-5e3d87b32b6f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.863339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf257-4a6a-4c6e-bd11-5e3d87b32b6f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.863549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cf257-4a6a-4c6e-bd11-5e3d87b32b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.865485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009141f-78b8-454b-897f-052b0009e549 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:02.881630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e06cf257-4a6a-4c6e-bd11-5e3d87b32b6f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:02.882826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e06cf257-4a6a-4c6e-bd11-5e3d87b32b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:02.900174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff623d-7f2f-4b84-b42f-ca381fc3ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:02.900814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff623d-7f2f-4b84-b42f-ca381fc3ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.901188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff623d-7f2f-4b84-b42f-ca381fc3ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.901573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff623d-7f2f-4b84-b42f-ca381fc3ebe2 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.902169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff623d-7f2f-4b84-b42f-ca381fc3ebe2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:02.902472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff623d-7f2f-4b84-b42f-ca381fc3ebe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:02.903850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff623d-7f2f-4b84-b42f-ca381fc3ebe2 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:02.911774 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 37/72] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:02.912552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff623d-7f2f-4b84-b42f-ca381fc3ebe2 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:02.948076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 36/73] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:02.949637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb37c5-a6e9-40c0-bb31-448e8087a1bd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:02.949923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb37c5-a6e9-40c0-bb31-448e8087a1bd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:02.950199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb37c5-a6e9-40c0-bb31-448e8087a1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:02.967478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-06bb37c5-a6e9-40c0-bb31-448e8087a1bd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:02.968650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-06bb37c5-a6e9-40c0-bb31-448e8087a1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:02.971377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:02.985479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:03.016996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff623d-7f2f-4b84-b42f-ca381fc3ebe2 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.046693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 37/74] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:03.047452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb37c5-a6e9-40c0-bb31-448e8087a1bd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.051940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5439806-663c-4eae-b6fb-8a421d6e2809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.052604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5439806-663c-4eae-b6fb-8a421d6e2809 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.057996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5439806-663c-4eae-b6fb-8a421d6e2809 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.057996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5439806-663c-4eae-b6fb-8a421d6e2809 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.074785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5439806-663c-4eae-b6fb-8a421d6e2809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-f5439806-663c-4eae-b6fb-8a421d6e2809 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:03.083269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:03.084092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:03.084840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:03.088936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 38/75] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:03.090756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5439806-663c-4eae-b6fb-8a421d6e2809 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.183115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb37c5-a6e9-40c0-bb31-448e8087a1bd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.223141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5439806-663c-4eae-b6fb-8a421d6e2809 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.226706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06a240-5ec6-4fdc-bbb6-49dcaf7d5613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.227311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06a240-5ec6-4fdc-bbb6-49dcaf7d5613 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.227589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06a240-5ec6-4fdc-bbb6-49dcaf7d5613 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.227860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06a240-5ec6-4fdc-bbb6-49dcaf7d5613 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.228517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06a240-5ec6-4fdc-bbb6-49dcaf7d5613 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:03.228841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06a240-5ec6-4fdc-bbb6-49dcaf7d5613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:03.231082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06a240-5ec6-4fdc-bbb6-49dcaf7d5613 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:03.243377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 38/76] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:03.248609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06a240-5ec6-4fdc-bbb6-49dcaf7d5613 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.251519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:03.256353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:03.262252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4affb5-389d-4685-8f98-aa7296e37597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.262887 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4affb5-389d-4685-8f98-aa7296e37597 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.263169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4affb5-389d-4685-8f98-aa7296e37597 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.266376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4affb5-389d-4685-8f98-aa7296e37597 None admin] PATH_INFO: `/v3/users/417da11974af471a8afca3fc24034882` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.278314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4affb5-389d-4685-8f98-aa7296e37597 None admin] RBAC: Authorizing `identity:get_user(user_id=417da11974af471a8afca3fc24034882)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:03.279795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4affb5-389d-4685-8f98-aa7296e37597 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:03.281670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 39/77] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/users/417da11974af471a8afca3fc24034882 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:03.285831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4affb5-389d-4685-8f98-aa7296e37597 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.358622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06a240-5ec6-4fdc-bbb6-49dcaf7d5613 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.395662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6b3a5-f9ba-4196-96e5-4a27c9cb7bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.396259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6b3a5-f9ba-4196-96e5-4a27c9cb7bba None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.396538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6b3a5-f9ba-4196-96e5-4a27c9cb7bba None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.396818 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6b3a5-f9ba-4196-96e5-4a27c9cb7bba None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.397484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6b3a5-f9ba-4196-96e5-4a27c9cb7bba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:03.397805 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6b3a5-f9ba-4196-96e5-4a27c9cb7bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:03.399175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6b3a5-f9ba-4196-96e5-4a27c9cb7bba None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:03.399977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4affb5-389d-4685-8f98-aa7296e37597 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.407614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 39/78] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:03.408477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6b3a5-f9ba-4196-96e5-4a27c9cb7bba None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.436984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786fdec-0e53-4af3-a178-a2e3db55fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.437636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786fdec-0e53-4af3-a178-a2e3db55fb6a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.437918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786fdec-0e53-4af3-a178-a2e3db55fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.438191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786fdec-0e53-4af3-a178-a2e3db55fb6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.447982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786fdec-0e53-4af3-a178-a2e3db55fb6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-d786fdec-0e53-4af3-a178-a2e3db55fb6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:03.452692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:03.453590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:03.454428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 40/79] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:03.455028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786fdec-0e53-4af3-a178-a2e3db55fb6a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.475233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:03.503971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:03.514557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6b3a5-f9ba-4196-96e5-4a27c9cb7bba None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.548443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f053f-2f12-42c5-8665-b4ef49b07fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.549112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f053f-2f12-42c5-8665-b4ef49b07fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.549395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f053f-2f12-42c5-8665-b4ef49b07fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.549670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f053f-2f12-42c5-8665-b4ef49b07fb0 None admin] PATH_INFO: `/v3/users/4fac3b46e24449e193279323e9d64fe0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.550796 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f053f-2f12-42c5-8665-b4ef49b07fb0 None admin] RBAC: Authorizing `identity:get_user(user_id=4fac3b46e24449e193279323e9d64fe0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:03.552257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f053f-2f12-42c5-8665-b4ef49b07fb0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:03.553728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786fdec-0e53-4af3-a178-a2e3db55fb6a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.558608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 40/80] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/users/4fac3b46e24449e193279323e9d64fe0 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:03.558608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f053f-2f12-42c5-8665-b4ef49b07fb0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.589478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d143e-9baf-44e6-9273-77397bdf14e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.590084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d143e-9baf-44e6-9273-77397bdf14e0 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.590359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d143e-9baf-44e6-9273-77397bdf14e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.590630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d143e-9baf-44e6-9273-77397bdf14e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.664656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f053f-2f12-42c5-8665-b4ef49b07fb0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.665808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d143e-9baf-44e6-9273-77397bdf14e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:03.667401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d143e-9baf-44e6-9273-77397bdf14e0 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:03.693459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:03.698849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e89e8-e65c-493f-8501-3b72ff7d6238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.699471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e89e8-e65c-493f-8501-3b72ff7d6238 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.699745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e89e8-e65c-493f-8501-3b72ff7d6238 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.700021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e89e8-e65c-493f-8501-3b72ff7d6238 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.701255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e89e8-e65c-493f-8501-3b72ff7d6238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:03.703907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e89e8-e65c-493f-8501-3b72ff7d6238 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:03.704336 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 41/81] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:03.709518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e89e8-e65c-493f-8501-3b72ff7d6238 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.715012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:03.765589 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 41/82] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:03.766789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d143e-9baf-44e6-9273-77397bdf14e0 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.809008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e89e8-e65c-493f-8501-3b72ff7d6238 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.842658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc3620-0682-4e96-947f-46d5e2af8c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.843298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc3620-0682-4e96-947f-46d5e2af8c43 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.843573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc3620-0682-4e96-947f-46d5e2af8c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.843847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc3620-0682-4e96-947f-46d5e2af8c43 None admin] PATH_INFO: `/v3/users/417da11974af471a8afca3fc24034882` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.845018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc3620-0682-4e96-947f-46d5e2af8c43 None admin] RBAC: Authorizing `identity:get_user(user_id=417da11974af471a8afca3fc24034882)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:03.846468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc3620-0682-4e96-947f-46d5e2af8c43 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:03.848051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 42/83] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/users/417da11974af471a8afca3fc24034882 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:03.848816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc3620-0682-4e96-947f-46d5e2af8c43 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.870255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d143e-9baf-44e6-9273-77397bdf14e0 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.903432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff6682-124b-453a-96c7-17e294f4e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.904033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff6682-124b-453a-96c7-17e294f4e626 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.904313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff6682-124b-453a-96c7-17e294f4e626 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.904583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff6682-124b-453a-96c7-17e294f4e626 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:03.905274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff6682-124b-453a-96c7-17e294f4e626 None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:03.905574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff6682-124b-453a-96c7-17e294f4e626 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:03.906966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff6682-124b-453a-96c7-17e294f4e626 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:03.914551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:03.914771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 42/84] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:03.915535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff6682-124b-453a-96c7-17e294f4e626 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:03.927369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:03.953061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc3620-0682-4e96-947f-46d5e2af8c43 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:03.986945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c988da-59e7-4ddd-b7c5-ff33a51d7cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:03.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c988da-59e7-4ddd-b7c5-ff33a51d7cce None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:03.987817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c988da-59e7-4ddd-b7c5-ff33a51d7cce None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:03.988087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c988da-59e7-4ddd-b7c5-ff33a51d7cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:04.014888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff6682-124b-453a-96c7-17e294f4e626 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:04.048477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7b7e5-7f68-42b4-a92c-cf45a796a438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:04.049106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7b7e5-7f68-42b4-a92c-cf45a796a438 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:04.049385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7b7e5-7f68-42b4-a92c-cf45a796a438 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:04.049654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7b7e5-7f68-42b4-a92c-cf45a796a438 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:04.050280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7b7e5-7f68-42b4-a92c-cf45a796a438 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:04.051715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7b7e5-7f68-42b4-a92c-cf45a796a438 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:04.053640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c988da-59e7-4ddd-b7c5-ff33a51d7cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:04.055088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c988da-59e7-4ddd-b7c5-ff33a51d7cce None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:04.125383 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:04.158798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 43/85] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:04.159700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c988da-59e7-4ddd-b7c5-ff33a51d7cce None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:04.170078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 43/86] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:03 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:04.171007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7b7e5-7f68-42b4-a92c-cf45a796a438 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:04.249835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:04.277772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c988da-59e7-4ddd-b7c5-ff33a51d7cce None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:04.291456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7b7e5-7f68-42b4-a92c-cf45a796a438 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:04.317777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458abb22-584b-4b9c-a03f-5d0fe094de08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:04.318414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458abb22-584b-4b9c-a03f-5d0fe094de08 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:04.318694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458abb22-584b-4b9c-a03f-5d0fe094de08 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:04.318962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458abb22-584b-4b9c-a03f-5d0fe094de08 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:04.320681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458abb22-584b-4b9c-a03f-5d0fe094de08 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:04.322353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458abb22-584b-4b9c-a03f-5d0fe094de08 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:04.324016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 44/87] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:04 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:04.324965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458abb22-584b-4b9c-a03f-5d0fe094de08 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:04.341491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87541d-6d65-4c85-b7eb-d66f1740f438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:04.342506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87541d-6d65-4c85-b7eb-d66f1740f438 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:04.342784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87541d-6d65-4c85-b7eb-d66f1740f438 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:04.343052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87541d-6d65-4c85-b7eb-d66f1740f438 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:04.366875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:04.424223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87541d-6d65-4c85-b7eb-d66f1740f438 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:04.426252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87541d-6d65-4c85-b7eb-d66f1740f438 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:04.451643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458abb22-584b-4b9c-a03f-5d0fe094de08 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:04.493300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22724595-7867-436a-900c-1b0d011d65e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:04.493942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22724595-7867-436a-900c-1b0d011d65e8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:04.494220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22724595-7867-436a-900c-1b0d011d65e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:04.494532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22724595-7867-436a-900c-1b0d011d65e8 None admin] PATH_INFO: `/v3/users/417da11974af471a8afca3fc24034882` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:04.496057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22724595-7867-436a-900c-1b0d011d65e8 None admin] RBAC: Authorizing `identity:get_user(user_id=417da11974af471a8afca3fc24034882)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:04.497548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22724595-7867-436a-900c-1b0d011d65e8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:04.499318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 45/88] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:04 2022] GET /identity/v3/users/417da11974af471a8afca3fc24034882 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:04.508630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22724595-7867-436a-900c-1b0d011d65e8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:04.508776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:04.542268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 44/89] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:04.547026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87541d-6d65-4c85-b7eb-d66f1740f438 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:04.668641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22724595-7867-436a-900c-1b0d011d65e8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:04.672117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:04.692552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87541d-6d65-4c85-b7eb-d66f1740f438 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:04.718228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fb9f1-3430-4be1-835e-1688f1dd462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:04.719286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fb9f1-3430-4be1-835e-1688f1dd462c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:04.719563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fb9f1-3430-4be1-835e-1688f1dd462c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:04.720107 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fb9f1-3430-4be1-835e-1688f1dd462c None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:04.721502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fb9f1-3430-4be1-835e-1688f1dd462c None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:04.723642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fb9f1-3430-4be1-835e-1688f1dd462c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:04.726369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 46/90] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:04 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:04.727216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fb9f1-3430-4be1-835e-1688f1dd462c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:04.735584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96178847-8fa9-4170-95a2-3bd39a385db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:04.736206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96178847-8fa9-4170-95a2-3bd39a385db1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:04.736486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96178847-8fa9-4170-95a2-3bd39a385db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:04.736932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96178847-8fa9-4170-95a2-3bd39a385db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:04.737712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96178847-8fa9-4170-95a2-3bd39a385db1 None admin] RBAC: Adding query filter params (role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c, user.id=417da11974af471a8afca3fc24034882) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:04.738026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96178847-8fa9-4170-95a2-3bd39a385db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:04.739700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96178847-8fa9-4170-95a2-3bd39a385db1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:04.746773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 45/91] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:04 2022] GET /identity/v3/role_assignments?user.id=417da11974af471a8afca3fc24034882&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:04.753775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96178847-8fa9-4170-95a2-3bd39a385db1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:04.805422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:04.885838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96178847-8fa9-4170-95a2-3bd39a385db1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:04.922606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fb9f1-3430-4be1-835e-1688f1dd462c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:04.923860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba434978-9c35-4b1a-82b3-997dfa670bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:04.924646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba434978-9c35-4b1a-82b3-997dfa670bbc None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:04.924962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba434978-9c35-4b1a-82b3-997dfa670bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:04.925237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba434978-9c35-4b1a-82b3-997dfa670bbc None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:04.926582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba434978-9c35-4b1a-82b3-997dfa670bbc None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:04.928141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba434978-9c35-4b1a-82b3-997dfa670bbc None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:04.929959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 46/92] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:04 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:04.930745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba434978-9c35-4b1a-82b3-997dfa670bbc None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:04.981558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947e822-35ae-4806-b0aa-8815d4669595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:04.982227 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947e822-35ae-4806-b0aa-8815d4669595 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:04.982502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947e822-35ae-4806-b0aa-8815d4669595 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:04.982778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947e822-35ae-4806-b0aa-8815d4669595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:05.043229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:05.047510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:05.051983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba434978-9c35-4b1a-82b3-997dfa670bbc None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:05.085295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947e822-35ae-4806-b0aa-8815d4669595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:05.086786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947e822-35ae-4806-b0aa-8815d4669595 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:05.093361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad323e0-8532-402a-b79a-bba9b013fadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:05.093967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad323e0-8532-402a-b79a-bba9b013fadb None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:05.094245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad323e0-8532-402a-b79a-bba9b013fadb None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:05.094517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad323e0-8532-402a-b79a-bba9b013fadb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:05.095806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad323e0-8532-402a-b79a-bba9b013fadb None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c, user.id=4fac3b46e24449e193279323e9d64fe0) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:05.096170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad323e0-8532-402a-b79a-bba9b013fadb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:05.098153 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad323e0-8532-402a-b79a-bba9b013fadb None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:05.106374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 47/93] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:04 2022] GET /identity/v3/role_assignments?user.id=4fac3b46e24449e193279323e9d64fe0&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:05.107580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad323e0-8532-402a-b79a-bba9b013fadb None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:05.234211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad323e0-8532-402a-b79a-bba9b013fadb None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:05.280257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab505cb-daf9-42a7-aae0-749dbe8a73a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:05.281170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab505cb-daf9-42a7-aae0-749dbe8a73a2 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:05.281390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab505cb-daf9-42a7-aae0-749dbe8a73a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:05.281594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab505cb-daf9-42a7-aae0-749dbe8a73a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:05.282349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab505cb-daf9-42a7-aae0-749dbe8a73a2 None admin] RBAC: Adding query filter params (role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c, user.id=417da11974af471a8afca3fc24034882) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:05.282657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab505cb-daf9-42a7-aae0-749dbe8a73a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:05.284144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab505cb-daf9-42a7-aae0-749dbe8a73a2 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:05.292426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 48/94] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:05 2022] GET /identity/v3/role_assignments?user.id=417da11974af471a8afca3fc24034882&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:05.346168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 47/95] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:05.352536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab505cb-daf9-42a7-aae0-749dbe8a73a2 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:05.399350 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:05.491289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab505cb-daf9-42a7-aae0-749dbe8a73a2 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:05.530564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28263c5f-5112-4b42-895b-d88d59214b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:05.531236 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28263c5f-5112-4b42-895b-d88d59214b69 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:05.531522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28263c5f-5112-4b42-895b-d88d59214b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:05.531789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28263c5f-5112-4b42-895b-d88d59214b69 None admin] PATH_INFO: `/v3/projects/2bd636be4e09401ca420cb21fd0e3a0f` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:05.533196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28263c5f-5112-4b42-895b-d88d59214b69 None admin] RBAC: Authorizing `identity:get_project(project_id=2bd636be4e09401ca420cb21fd0e3a0f)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:05.534618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28263c5f-5112-4b42-895b-d88d59214b69 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:05.536413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 49/96] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:05 2022] GET /identity/v3/projects/2bd636be4e09401ca420cb21fd0e3a0f => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:05.545931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947e822-35ae-4806-b0aa-8815d4669595 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:05.669392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:05.701775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947e822-35ae-4806-b0aa-8815d4669595 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:05.750192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61feb4-f990-488c-8d00-c03b80a00034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:05.750819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61feb4-f990-488c-8d00-c03b80a00034 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:05.751100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61feb4-f990-488c-8d00-c03b80a00034 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:05.751375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61feb4-f990-488c-8d00-c03b80a00034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:05.752115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61feb4-f990-488c-8d00-c03b80a00034 None admin] RBAC: Adding query filter params (role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=2bd636be4e09401ca420cb21fd0e3a0f, user.id=417da11974af471a8afca3fc24034882) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:05.752422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61feb4-f990-488c-8d00-c03b80a00034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:05.753969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61feb4-f990-488c-8d00-c03b80a00034 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:05.763659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 48/97] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:05 2022] GET /identity/v3/role_assignments?user.id=417da11974af471a8afca3fc24034882&scope.project.id=2bd636be4e09401ca420cb21fd0e3a0f&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:06.757739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e238d3e-4d56-411a-bec8-a9218bec69fd None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:06.758247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e238d3e-4d56-411a-bec8-a9218bec69fd None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:06.758320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e238d3e-4d56-411a-bec8-a9218bec69fd None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:06.759640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 50/98] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:06.765517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac7daf-76af-49b6-929c-ec1d39569b4a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:06.765979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac7daf-76af-49b6-929c-ec1d39569b4a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:06.766055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac7daf-76af-49b6-929c-ec1d39569b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:06.790055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8aac7daf-76af-49b6-929c-ec1d39569b4a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:06.791248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8aac7daf-76af-49b6-929c-ec1d39569b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:06.885644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 49/99] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:06.907931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2a094-0725-494e-9893-f24ef58d298e None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:06.908190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2a094-0725-494e-9893-f24ef58d298e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:06.908410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2a094-0725-494e-9893-f24ef58d298e None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:06.909789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 51/100] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:06.915521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ece667-9c6d-4b2d-9dac-af37707dc76d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:06.915816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ece667-9c6d-4b2d-9dac-af37707dc76d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:06.916079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ece667-9c6d-4b2d-9dac-af37707dc76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:06.944894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-15ece667-9c6d-4b2d-9dac-af37707dc76d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:06.946070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-15ece667-9c6d-4b2d-9dac-af37707dc76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:07.026373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:07.034624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 50/101] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:07.272766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2a094-0725-494e-9893-f24ef58d298e None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:07.402942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:07.411464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2a094-0725-494e-9893-f24ef58d298e None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:07.460742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698ced9-f107-444f-a578-1bce4e81acd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:07.461735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698ced9-f107-444f-a578-1bce4e81acd1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:07.462227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698ced9-f107-444f-a578-1bce4e81acd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:07.462713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698ced9-f107-444f-a578-1bce4e81acd1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:07.476682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698ced9-f107-444f-a578-1bce4e81acd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-d698ced9-f107-444f-a578-1bce4e81acd1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:07.482195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:07.483026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:07.483752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:07.485986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 52/102] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:11:07.492139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ece667-9c6d-4b2d-9dac-af37707dc76d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:07.525520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698ced9-f107-444f-a578-1bce4e81acd1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:07.716263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ece667-9c6d-4b2d-9dac-af37707dc76d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:07.755179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698ced9-f107-444f-a578-1bce4e81acd1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:07.776350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dd37a-93e6-4edc-bd49-4a5ed990753f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:07.777266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dd37a-93e6-4edc-bd49-4a5ed990753f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:07.777555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dd37a-93e6-4edc-bd49-4a5ed990753f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:07.777835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dd37a-93e6-4edc-bd49-4a5ed990753f None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:07.778802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dd37a-93e6-4edc-bd49-4a5ed990753f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:07.779118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dd37a-93e6-4edc-bd49-4a5ed990753f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:07.780855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dd37a-93e6-4edc-bd49-4a5ed990753f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:07.797939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 51/103] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:07.799608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2895683-252e-4da7-8938-34f4150c4dfb None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:07.800092 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2895683-252e-4da7-8938-34f4150c4dfb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:07.800606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2895683-252e-4da7-8938-34f4150c4dfb None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:07.802038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 52/104] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:07.806769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2895683-252e-4da7-8938-34f4150c4dfb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:07.827391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c3ec6-ebd1-4f7a-beb6-c0704c8813d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:07.828025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c3ec6-ebd1-4f7a-beb6-c0704c8813d7 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:07.828292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c3ec6-ebd1-4f7a-beb6-c0704c8813d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:07.828564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c3ec6-ebd1-4f7a-beb6-c0704c8813d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:07.829779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c3ec6-ebd1-4f7a-beb6-c0704c8813d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:07.831225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c3ec6-ebd1-4f7a-beb6-c0704c8813d7 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:07.833055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 53/105] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:07.834285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d221f-b08f-479a-80d6-ac80e7cafd5a None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:07.834718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d221f-b08f-479a-80d6-ac80e7cafd5a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:07.835125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d221f-b08f-479a-80d6-ac80e7cafd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:07.855103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ef3d221f-b08f-479a-80d6-ac80e7cafd5a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:07.856928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ef3d221f-b08f-479a-80d6-ac80e7cafd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:07.874562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:07.898857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:07.929741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 54/106] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:07.930895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d221f-b08f-479a-80d6-ac80e7cafd5a None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:07.986470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2895683-252e-4da7-8938-34f4150c4dfb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:08.080966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d221f-b08f-479a-80d6-ac80e7cafd5a None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:08.101521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bfb27-7dfb-4836-abb4-2e96baf5d208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:08.102182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bfb27-7dfb-4836-abb4-2e96baf5d208 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.102464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bfb27-7dfb-4836-abb4-2e96baf5d208 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.102737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bfb27-7dfb-4836-abb4-2e96baf5d208 None admin] PATH_INFO: `/v3/users/ea009dc138d6487b9145db2c059d7a1f` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.140483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39395b4-d09e-43a1-8aa0-8c4b6d559106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:08.141500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bfb27-7dfb-4836-abb4-2e96baf5d208 None admin] RBAC: Authorizing `identity:get_user(user_id=ea009dc138d6487b9145db2c059d7a1f)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:08.141726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39395b4-d09e-43a1-8aa0-8c4b6d559106 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.142382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39395b4-d09e-43a1-8aa0-8c4b6d559106 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.142893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39395b4-d09e-43a1-8aa0-8c4b6d559106 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.143126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bfb27-7dfb-4836-abb4-2e96baf5d208 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:08.144018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39395b4-d09e-43a1-8aa0-8c4b6d559106 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:08.145972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 53/107] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:07 2022] GET /identity/v3/users/ea009dc138d6487b9145db2c059d7a1f => generated 289 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:08.147012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39395b4-d09e-43a1-8aa0-8c4b6d559106 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:08.157929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9afada-95ff-4355-9ab3-33b51070a930 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.157929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9afada-95ff-4355-9ab3-33b51070a930 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.157929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9afada-95ff-4355-9ab3-33b51070a930 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.158233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 54/108] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:11:08.161654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9afada-95ff-4355-9ab3-33b51070a930 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:08.282749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 55/109] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:11:07 2022] POST /identity/v3/projects => generated 302 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:08.284470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8b004-0329-43e6-baa3-d7d638aa607b None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.285122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8b004-0329-43e6-baa3-d7d638aa607b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.285593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8b004-0329-43e6-baa3-d7d638aa607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.290879 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9afada-95ff-4355-9ab3-33b51070a930 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:08.309214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:08.319909 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-71d8b004-0329-43e6-baa3-d7d638aa607b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:08.324433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-71d8b004-0329-43e6-baa3-d7d638aa607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:08.335189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:08.371077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a0d09-2654-4eab-85fa-704f1c99a682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:08.372133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a0d09-2654-4eab-85fa-704f1c99a682 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.372627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a0d09-2654-4eab-85fa-704f1c99a682 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.373224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a0d09-2654-4eab-85fa-704f1c99a682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.454332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a0d09-2654-4eab-85fa-704f1c99a682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:08.456224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a0d09-2654-4eab-85fa-704f1c99a682 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:08.461692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 56/110] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:08.462377 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8b004-0329-43e6-baa3-d7d638aa607b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:08.588994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 55/111] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:08.591076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209c692-d90e-4d08-9aeb-042117fa6f7f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.591579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209c692-d90e-4d08-9aeb-042117fa6f7f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.592044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209c692-d90e-4d08-9aeb-042117fa6f7f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.593515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 56/112] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:08.595306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209c692-d90e-4d08-9aeb-042117fa6f7f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:08.616399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:08.643203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8b004-0329-43e6-baa3-d7d638aa607b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:08.734108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb7c3c-6da5-4cda-9777-b179f34e6603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:08.734108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb7c3c-6da5-4cda-9777-b179f34e6603 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.734108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb7c3c-6da5-4cda-9777-b179f34e6603 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.734108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb7c3c-6da5-4cda-9777-b179f34e6603 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb7c3c-6da5-4cda-9777-b179f34e6603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-66bb7c3c-6da5-4cda-9777-b179f34e6603 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:08.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:08.735664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:08.736459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 57/113] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:08.737320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b0b3a-cff0-42c5-850e-313f20c2fca8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.737320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b0b3a-cff0-42c5-850e-313f20c2fca8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.737320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b0b3a-cff0-42c5-850e-313f20c2fca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.737320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ce1b0b3a-cff0-42c5-850e-313f20c2fca8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:08.737591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209c692-d90e-4d08-9aeb-042117fa6f7f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:08.739568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ce1b0b3a-cff0-42c5-850e-313f20c2fca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:08.779529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839ce9b-7277-4009-9f00-8bdbf7d7a8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:08.780226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839ce9b-7277-4009-9f00-8bdbf7d7a8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.780442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839ce9b-7277-4009-9f00-8bdbf7d7a8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.780714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839ce9b-7277-4009-9f00-8bdbf7d7a8d9 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.781928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839ce9b-7277-4009-9f00-8bdbf7d7a8d9 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:08.783391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839ce9b-7277-4009-9f00-8bdbf7d7a8d9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:08.785552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 57/114] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:08 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:08.786700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b89db9-f714-42af-bed0-db6f746a170e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.789208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b89db9-f714-42af-bed0-db6f746a170e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.789700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b89db9-f714-42af-bed0-db6f746a170e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.791943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 58/115] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:08.793641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b89db9-f714-42af-bed0-db6f746a170e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:08.820309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 58/116] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:08.826366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b0b3a-cff0-42c5-850e-313f20c2fca8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:08.890726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:08.909379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b89db9-f714-42af-bed0-db6f746a170e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:08.945439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c096c-8ffc-4ffd-93eb-b1b06315f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:08.946051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c096c-8ffc-4ffd-93eb-b1b06315f13f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:08.946325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c096c-8ffc-4ffd-93eb-b1b06315f13f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:08.946595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c096c-8ffc-4ffd-93eb-b1b06315f13f None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:08.947283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c096c-8ffc-4ffd-93eb-b1b06315f13f None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:08.948412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c096c-8ffc-4ffd-93eb-b1b06315f13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:08.949152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c096c-8ffc-4ffd-93eb-b1b06315f13f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:08.960590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 59/117] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:08.961667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c096c-8ffc-4ffd-93eb-b1b06315f13f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:08.982118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b0b3a-cff0-42c5-850e-313f20c2fca8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.025208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:09.033875 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae516a-0b4b-4ef6-a434-f4ea35b8a997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:09.043303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae516a-0b4b-4ef6-a434-f4ea35b8a997 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.043303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae516a-0b4b-4ef6-a434-f4ea35b8a997 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.043303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae516a-0b4b-4ef6-a434-f4ea35b8a997 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.059037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae516a-0b4b-4ef6-a434-f4ea35b8a997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-20ae516a-0b4b-4ef6-a434-f4ea35b8a997 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:09.066171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:09.067241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:09.068010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:09.070868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 59/118] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:11:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:09.072372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4eab2-43d0-4552-b744-a91e58e37347 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.073208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4eab2-43d0-4552-b744-a91e58e37347 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.073663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4eab2-43d0-4552-b744-a91e58e37347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.088985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c096c-8ffc-4ffd-93eb-b1b06315f13f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.096289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-7da4eab2-43d0-4552-b744-a91e58e37347 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:09.098177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7da4eab2-43d0-4552-b744-a91e58e37347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:09.127555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b27f99-4fa3-47c4-b119-9175a38683bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:09.128159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b27f99-4fa3-47c4-b119-9175a38683bc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.128442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b27f99-4fa3-47c4-b119-9175a38683bc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.128711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b27f99-4fa3-47c4-b119-9175a38683bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.129457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b27f99-4fa3-47c4-b119-9175a38683bc None admin] RBAC: Adding query filter params (role.id=c54d8e4858914575b2ba91acf3233251, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c, user.id=ea009dc138d6487b9145db2c059d7a1f) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:09.129761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b27f99-4fa3-47c4-b119-9175a38683bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:09.131186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b27f99-4fa3-47c4-b119-9175a38683bc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:09.144907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 60/119] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:08 2022] GET /identity/v3/role_assignments?user.id=ea009dc138d6487b9145db2c059d7a1f&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=c54d8e4858914575b2ba91acf3233251 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:09.144907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b27f99-4fa3-47c4-b119-9175a38683bc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:09.158263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:09.190737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 60/120] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:09.191840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4eab2-43d0-4552-b744-a91e58e37347 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:09.327446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b27f99-4fa3-47c4-b119-9175a38683bc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.351139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4eab2-43d0-4552-b744-a91e58e37347 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.381761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:09.384928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2719b1-7b63-4bc0-8658-bfee26a8a0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:09.385966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2719b1-7b63-4bc0-8658-bfee26a8a0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.386245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2719b1-7b63-4bc0-8658-bfee26a8a0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.386516 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2719b1-7b63-4bc0-8658-bfee26a8a0f8 None admin] PATH_INFO: `/v3/users/417da11974af471a8afca3fc24034882` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.389570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2719b1-7b63-4bc0-8658-bfee26a8a0f8 None admin] RBAC: Authorizing `identity:get_user(user_id=417da11974af471a8afca3fc24034882)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:09.393170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2719b1-7b63-4bc0-8658-bfee26a8a0f8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:09.393170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 61/121] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:09 2022] GET /identity/v3/users/417da11974af471a8afca3fc24034882 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:09.393474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2719b1-7b63-4bc0-8658-bfee26a8a0f8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:09.418763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898cb4f-285a-4853-8c4c-03f6363ad03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:09.419304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898cb4f-285a-4853-8c4c-03f6363ad03f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.419582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898cb4f-285a-4853-8c4c-03f6363ad03f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.419852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898cb4f-285a-4853-8c4c-03f6363ad03f None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.420528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898cb4f-285a-4853-8c4c-03f6363ad03f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:09.420857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898cb4f-285a-4853-8c4c-03f6363ad03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:09.422251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898cb4f-285a-4853-8c4c-03f6363ad03f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:09.439147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 61/122] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:09.439992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898cb4f-285a-4853-8c4c-03f6363ad03f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:09.516423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:09.538636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2719b1-7b63-4bc0-8658-bfee26a8a0f8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.585208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ebb2c-0917-4fcb-9a9a-a996b4a87d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:09.585855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ebb2c-0917-4fcb-9a9a-a996b4a87d0d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.586417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ebb2c-0917-4fcb-9a9a-a996b4a87d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.586695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ebb2c-0917-4fcb-9a9a-a996b4a87d0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.600747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ebb2c-0917-4fcb-9a9a-a996b4a87d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-4d7ebb2c-0917-4fcb-9a9a-a996b4a87d0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:09.608584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:09.609474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:09.610804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 62/123] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:09.611762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ebb2c-0917-4fcb-9a9a-a996b4a87d0d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:09.629754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898cb4f-285a-4853-8c4c-03f6363ad03f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.680763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b560925-f8e7-4996-895b-2fc3566372f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:09.681401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b560925-f8e7-4996-895b-2fc3566372f4 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.681684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b560925-f8e7-4996-895b-2fc3566372f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.682224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b560925-f8e7-4996-895b-2fc3566372f4 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.683771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b560925-f8e7-4996-895b-2fc3566372f4 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:09.686203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b560925-f8e7-4996-895b-2fc3566372f4 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:09.689076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 62/124] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:09 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:09.690538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b560925-f8e7-4996-895b-2fc3566372f4 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:09.753295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:09.759098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ebb2c-0917-4fcb-9a9a-a996b4a87d0d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.794919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ad04e-6daa-4769-94a0-cccbab2dcc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:09.795553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ad04e-6daa-4769-94a0-cccbab2dcc4b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.795812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ad04e-6daa-4769-94a0-cccbab2dcc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.796082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ad04e-6daa-4769-94a0-cccbab2dcc4b None admin] PATH_INFO: `/v3/users/4fac3b46e24449e193279323e9d64fe0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.797221 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ad04e-6daa-4769-94a0-cccbab2dcc4b None admin] RBAC: Authorizing `identity:get_user(user_id=4fac3b46e24449e193279323e9d64fe0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:09.798665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ad04e-6daa-4769-94a0-cccbab2dcc4b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:09.800223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 63/125] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:09 2022] GET /identity/v3/users/4fac3b46e24449e193279323e9d64fe0 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:09.805062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ad04e-6daa-4769-94a0-cccbab2dcc4b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:09.813559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b560925-f8e7-4996-895b-2fc3566372f4 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.814229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:09.851512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b199b-8ceb-4765-b434-0c7a450a520d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:09.852137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b199b-8ceb-4765-b434-0c7a450a520d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.852413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b199b-8ceb-4765-b434-0c7a450a520d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.852685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b199b-8ceb-4765-b434-0c7a450a520d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.864962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b199b-8ceb-4765-b434-0c7a450a520d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-933b199b-8ceb-4765-b434-0c7a450a520d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:09.872525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:09.874940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 63/126] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:09.876964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b199b-8ceb-4765-b434-0c7a450a520d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:09.908967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ad04e-6daa-4769-94a0-cccbab2dcc4b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.943403 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639afeb-7445-4008-9313-e44c0e831e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:09.943932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639afeb-7445-4008-9313-e44c0e831e57 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:09.944208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639afeb-7445-4008-9313-e44c0e831e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:09.944579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639afeb-7445-4008-9313-e44c0e831e57 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:09.945368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639afeb-7445-4008-9313-e44c0e831e57 None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:09.945673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639afeb-7445-4008-9313-e44c0e831e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:09.947100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1639afeb-7445-4008-9313-e44c0e831e57 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:09.956713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 64/127] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:09.957387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639afeb-7445-4008-9313-e44c0e831e57 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:09.987813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b199b-8ceb-4765-b434-0c7a450a520d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:09.990229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:10.023664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d437d-c69d-4ca1-800d-ffdc22bd79ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:10.024321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d437d-c69d-4ca1-800d-ffdc22bd79ac None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.024599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d437d-c69d-4ca1-800d-ffdc22bd79ac None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.024894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d437d-c69d-4ca1-800d-ffdc22bd79ac None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/users/417da11974af471a8afca3fc24034882/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.028463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d437d-c69d-4ca1-800d-ffdc22bd79ac None admin] RBAC: Authorizing `identity:create_grant(project_id=2eee6585d6f04caf9d4244aa86bb7c0c, user_id=417da11974af471a8afca3fc24034882, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:10.029957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d437d-c69d-4ca1-800d-ffdc22bd79ac None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:10.033331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:10.066747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639afeb-7445-4008-9313-e44c0e831e57 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:10.102073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b978ec-75fc-4401-b908-10c67d4fb4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:10.102929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b978ec-75fc-4401-b908-10c67d4fb4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.103417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b978ec-75fc-4401-b908-10c67d4fb4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.104584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b978ec-75fc-4401-b908-10c67d4fb4f5 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.106186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b978ec-75fc-4401-b908-10c67d4fb4f5 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:10.107834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b978ec-75fc-4401-b908-10c67d4fb4f5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:10.109634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 65/128] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:09 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:10.110539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b978ec-75fc-4401-b908-10c67d4fb4f5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:10.128107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 64/129] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:11:09 2022] PUT /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/users/417da11974af471a8afca3fc24034882/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:10.128107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d437d-c69d-4ca1-800d-ffdc22bd79ac None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:10.256335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d437d-c69d-4ca1-800d-ffdc22bd79ac None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:10.298085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bd5b4-ea92-4e84-a6f9-4eb982c194cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:10.299054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bd5b4-ea92-4e84-a6f9-4eb982c194cf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.299334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bd5b4-ea92-4e84-a6f9-4eb982c194cf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.299603 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bd5b4-ea92-4e84-a6f9-4eb982c194cf None admin] PATH_INFO: `/v3/users/417da11974af471a8afca3fc24034882` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.300876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bd5b4-ea92-4e84-a6f9-4eb982c194cf None admin] RBAC: Authorizing `identity:get_user(user_id=417da11974af471a8afca3fc24034882)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:10.303007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bd5b4-ea92-4e84-a6f9-4eb982c194cf None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:10.304703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 65/130] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:10 2022] GET /identity/v3/users/417da11974af471a8afca3fc24034882 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:10.306321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bd5b4-ea92-4e84-a6f9-4eb982c194cf None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:10.342650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:10.348186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b978ec-75fc-4401-b908-10c67d4fb4f5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:10.394941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32ce11-924c-48be-9bfa-3ecae826ce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:10.395939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32ce11-924c-48be-9bfa-3ecae826ce7d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.396413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32ce11-924c-48be-9bfa-3ecae826ce7d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.396979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32ce11-924c-48be-9bfa-3ecae826ce7d None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.397842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32ce11-924c-48be-9bfa-3ecae826ce7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:10.398342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32ce11-924c-48be-9bfa-3ecae826ce7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:10.399962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32ce11-924c-48be-9bfa-3ecae826ce7d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:10.403531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:10.411558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 66/131] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:10.412950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32ce11-924c-48be-9bfa-3ecae826ce7d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:10.445886 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bd5b4-ea92-4e84-a6f9-4eb982c194cf None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:10.516276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf66dc6-81d1-4c1d-90e1-e63dabe756cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:10.517411 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf66dc6-81d1-4c1d-90e1-e63dabe756cd None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.518564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf66dc6-81d1-4c1d-90e1-e63dabe756cd None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.519197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf66dc6-81d1-4c1d-90e1-e63dabe756cd None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/users/4fac3b46e24449e193279323e9d64fe0/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.523366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf66dc6-81d1-4c1d-90e1-e63dabe756cd None admin] RBAC: Authorizing `identity:create_grant(project_id=2eee6585d6f04caf9d4244aa86bb7c0c, user_id=4fac3b46e24449e193279323e9d64fe0, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:10.525287 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf66dc6-81d1-4c1d-90e1-e63dabe756cd None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:10.549192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32ce11-924c-48be-9bfa-3ecae826ce7d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:10.587495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec54fe6-d7e3-4d38-b64c-8f7eb7b842b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:10.588601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec54fe6-d7e3-4d38-b64c-8f7eb7b842b6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.589121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec54fe6-d7e3-4d38-b64c-8f7eb7b842b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.589601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec54fe6-d7e3-4d38-b64c-8f7eb7b842b6 None admin] PATH_INFO: `/v3/projects/2bd636be4e09401ca420cb21fd0e3a0f` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.591038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec54fe6-d7e3-4d38-b64c-8f7eb7b842b6 None admin] RBAC: Authorizing `identity:get_project(project_id=2bd636be4e09401ca420cb21fd0e3a0f)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:10.592662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec54fe6-d7e3-4d38-b64c-8f7eb7b842b6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:10.594519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 67/132] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:10 2022] GET /identity/v3/projects/2bd636be4e09401ca420cb21fd0e3a0f => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:10.595146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec54fe6-d7e3-4d38-b64c-8f7eb7b842b6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:10.621622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 66/133] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:11:10 2022] PUT /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/users/4fac3b46e24449e193279323e9d64fe0/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:10.622875 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97741fe-34c2-491d-ac2c-04ee159e6db3 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.623158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97741fe-34c2-491d-ac2c-04ee159e6db3 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.623425 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97741fe-34c2-491d-ac2c-04ee159e6db3 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.626145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 67/134] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:10.626145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97741fe-34c2-491d-ac2c-04ee159e6db3 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:10.688486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:10.732534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec54fe6-d7e3-4d38-b64c-8f7eb7b842b6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:10.774982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62e026-ca80-4c59-abcf-c36169537437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:10.776159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62e026-ca80-4c59-abcf-c36169537437 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.776425 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62e026-ca80-4c59-abcf-c36169537437 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.776665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62e026-ca80-4c59-abcf-c36169537437 None admin] PATH_INFO: `/v3/users/417da11974af471a8afca3fc24034882` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.777882 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62e026-ca80-4c59-abcf-c36169537437 None admin] RBAC: Authorizing `identity:get_user(user_id=417da11974af471a8afca3fc24034882)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:10.779577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62e026-ca80-4c59-abcf-c36169537437 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:10.781582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 68/135] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:10 2022] GET /identity/v3/users/417da11974af471a8afca3fc24034882 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:10.782792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d2e07-9681-42fb-a401-54fe5e2110d7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.783073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d2e07-9681-42fb-a401-54fe5e2110d7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.783346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d2e07-9681-42fb-a401-54fe5e2110d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.807800 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-391d2e07-9681-42fb-a401-54fe5e2110d7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:10.809017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-391d2e07-9681-42fb-a401-54fe5e2110d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:10.823284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:10.842799 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97741fe-34c2-491d-ac2c-04ee159e6db3 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:10.898428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 69/136] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:10.898949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2169e0-95a3-4665-ba38-39c3fbb403c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:10.899688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2169e0-95a3-4665-ba38-39c3fbb403c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:10.900019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d2e07-9681-42fb-a401-54fe5e2110d7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:10.900120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2169e0-95a3-4665-ba38-39c3fbb403c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:10.900380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2169e0-95a3-4665-ba38-39c3fbb403c9 None admin] PATH_INFO: `/v3/projects/2bd636be4e09401ca420cb21fd0e3a0f/users/417da11974af471a8afca3fc24034882/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:10.902947 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2169e0-95a3-4665-ba38-39c3fbb403c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bd636be4e09401ca420cb21fd0e3a0f, user_id=417da11974af471a8afca3fc24034882, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:10.905503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2169e0-95a3-4665-ba38-39c3fbb403c9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:10.976244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 68/137] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:11:10 2022] PUT /identity/v3/projects/2bd636be4e09401ca420cb21fd0e3a0f/users/417da11974af471a8afca3fc24034882/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:10.986733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:11.051886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d2e07-9681-42fb-a401-54fe5e2110d7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:11.092369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1ecc3-fd25-447c-923a-dd071ed533d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:11.093043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1ecc3-fd25-447c-923a-dd071ed533d0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:11.093317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1ecc3-fd25-447c-923a-dd071ed533d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:11.093643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1ecc3-fd25-447c-923a-dd071ed533d0 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:11.094864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1ecc3-fd25-447c-923a-dd071ed533d0 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:11.096367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1ecc3-fd25-447c-923a-dd071ed533d0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:11.098206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 70/138] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:10 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:11.106229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2169e0-95a3-4665-ba38-39c3fbb403c9 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:11.238618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2169e0-95a3-4665-ba38-39c3fbb403c9 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:11.283146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66a634-70ae-44a4-ba8f-8e9c7f1c5199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:11.283808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66a634-70ae-44a4-ba8f-8e9c7f1c5199 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:11.284085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66a634-70ae-44a4-ba8f-8e9c7f1c5199 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:11.284353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66a634-70ae-44a4-ba8f-8e9c7f1c5199 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/users/417da11974af471a8afca3fc24034882/roles/1fbe1ab7a2864ba1818c7038baf57c34` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:11.286760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66a634-70ae-44a4-ba8f-8e9c7f1c5199 None admin] RBAC: Authorizing `identity:create_grant(project_id=2eee6585d6f04caf9d4244aa86bb7c0c, user_id=417da11974af471a8afca3fc24034882, role_id=1fbe1ab7a2864ba1818c7038baf57c34)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:11.288227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66a634-70ae-44a4-ba8f-8e9c7f1c5199 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:11.359142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1ecc3-fd25-447c-923a-dd071ed533d0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:11.378618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 69/139] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:11:11 2022] PUT /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/users/417da11974af471a8afca3fc24034882/roles/1fbe1ab7a2864ba1818c7038baf57c34 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:11.386032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cea630-3475-4a86-8433-3137c95d1750 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:11.386032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cea630-3475-4a86-8433-3137c95d1750 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:11.386032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cea630-3475-4a86-8433-3137c95d1750 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:11.386750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 70/140] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:11.397169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a78940-1666-4c9a-b731-6bbc1d28d6f5 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:11.397804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a78940-1666-4c9a-b731-6bbc1d28d6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:11.398282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a78940-1666-4c9a-b731-6bbc1d28d6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:11.426802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-06a78940-1666-4c9a-b731-6bbc1d28d6f5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:11.428376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-06a78940-1666-4c9a-b731-6bbc1d28d6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:11.442911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:11.498154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1ecc3-fd25-447c-923a-dd071ed533d0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:11.537189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:11.545594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51972d-9941-4305-9737-d878c8300783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:11.546253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51972d-9941-4305-9737-d878c8300783 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:11.546548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51972d-9941-4305-9737-d878c8300783 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:11.546885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51972d-9941-4305-9737-d878c8300783 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:11.548194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51972d-9941-4305-9737-d878c8300783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:11.549955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee51972d-9941-4305-9737-d878c8300783 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:11.552574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 71/141] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:11.559227 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51972d-9941-4305-9737-d878c8300783 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:11.610952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 71/142] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:11.681426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51972d-9941-4305-9737-d878c8300783 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:11.742508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d171170-6b5f-4817-9a6a-ab362e429e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:11.743234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d171170-6b5f-4817-9a6a-ab362e429e22 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:11.743562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d171170-6b5f-4817-9a6a-ab362e429e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:11.743837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d171170-6b5f-4817-9a6a-ab362e429e22 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:11.745151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d171170-6b5f-4817-9a6a-ab362e429e22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:11.746619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d171170-6b5f-4817-9a6a-ab362e429e22 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:11.866185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 72/143] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:11 2022] POST /identity/v3/users => generated 283 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:11:12.108426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a78940-1666-4c9a-b731-6bbc1d28d6f5 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:12.228697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a78940-1666-4c9a-b731-6bbc1d28d6f5 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:12.275028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd3244-c60e-4c25-b7da-9d0f565769d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:12.275697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd3244-c60e-4c25-b7da-9d0f565769d1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:12.276041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd3244-c60e-4c25-b7da-9d0f565769d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:12.276311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd3244-c60e-4c25-b7da-9d0f565769d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:12.289752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd3244-c60e-4c25-b7da-9d0f565769d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-f9cd3244-c60e-4c25-b7da-9d0f565769d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:12.294704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:12.295542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:12.297401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 72/144] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:12.301539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d171170-6b5f-4817-9a6a-ab362e429e22 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:12.373507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:12.430578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d171170-6b5f-4817-9a6a-ab362e429e22 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:12.475808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b300623-6a7a-4e84-9aa0-5dafd8f2c228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:12.476777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b300623-6a7a-4e84-9aa0-5dafd8f2c228 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:12.477268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b300623-6a7a-4e84-9aa0-5dafd8f2c228 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:12.486043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b300623-6a7a-4e84-9aa0-5dafd8f2c228 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:12.486043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b300623-6a7a-4e84-9aa0-5dafd8f2c228 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:12.486043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b300623-6a7a-4e84-9aa0-5dafd8f2c228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:12.486043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b300623-6a7a-4e84-9aa0-5dafd8f2c228 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:12.493028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 73/145] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:12.499528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd3244-c60e-4c25-b7da-9d0f565769d1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:12.503043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:12.641387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd3244-c60e-4c25-b7da-9d0f565769d1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:12.682570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e1916-65e3-4319-b104-54cb3a050788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:12.683271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e1916-65e3-4319-b104-54cb3a050788 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:12.683551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e1916-65e3-4319-b104-54cb3a050788 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:12.683820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e1916-65e3-4319-b104-54cb3a050788 None admin] PATH_INFO: `/v3/users/ea009dc138d6487b9145db2c059d7a1f` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:12.685002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e1916-65e3-4319-b104-54cb3a050788 None admin] RBAC: Authorizing `identity:get_user(user_id=ea009dc138d6487b9145db2c059d7a1f)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:12.686451 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e1916-65e3-4319-b104-54cb3a050788 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:12.688043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 73/146] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:12 2022] GET /identity/v3/users/ea009dc138d6487b9145db2c059d7a1f => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:12.694203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b300623-6a7a-4e84-9aa0-5dafd8f2c228 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:12.833224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b300623-6a7a-4e84-9aa0-5dafd8f2c228 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:12.858888 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:12.879143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73598639-9c1e-4312-9d9f-ec2c11b6daaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:12.880447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73598639-9c1e-4312-9d9f-ec2c11b6daaa None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:12.881045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73598639-9c1e-4312-9d9f-ec2c11b6daaa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:12.881461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73598639-9c1e-4312-9d9f-ec2c11b6daaa None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:12.886094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73598639-9c1e-4312-9d9f-ec2c11b6daaa None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:12.887890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73598639-9c1e-4312-9d9f-ec2c11b6daaa None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:12.890092 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 74/147] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:12 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:12.896112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e1916-65e3-4319-b104-54cb3a050788 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:12.957112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acaea90-fdd3-4ffb-97f0-5bd785ca6a49 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:12.957645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acaea90-fdd3-4ffb-97f0-5bd785ca6a49 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:12.958059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acaea90-fdd3-4ffb-97f0-5bd785ca6a49 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:12.959567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 75/148] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:12.965635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a8978-68f7-4509-98ee-c4bf69004b10 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:12.966107 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a8978-68f7-4509-98ee-c4bf69004b10 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:12.966537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a8978-68f7-4509-98ee-c4bf69004b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:12.997279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:13.005800 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-299a8978-68f7-4509-98ee-c4bf69004b10 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:13.007196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-299a8978-68f7-4509-98ee-c4bf69004b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:13.043521 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e1916-65e3-4319-b104-54cb3a050788 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:13.092922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532949-bffe-4673-b93e-8a361fa52eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:13.093524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532949-bffe-4673-b93e-8a361fa52eea None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:13.093859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532949-bffe-4673-b93e-8a361fa52eea None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:13.094137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532949-bffe-4673-b93e-8a361fa52eea None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/users/ea009dc138d6487b9145db2c059d7a1f/roles/c54d8e4858914575b2ba91acf3233251` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:13.096352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c532949-bffe-4673-b93e-8a361fa52eea None admin] RBAC: Authorizing `identity:create_grant(project_id=2eee6585d6f04caf9d4244aa86bb7c0c, user_id=ea009dc138d6487b9145db2c059d7a1f, role_id=c54d8e4858914575b2ba91acf3233251)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:13.098359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c532949-bffe-4673-b93e-8a361fa52eea None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:13.178096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 76/149] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:13.200707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 74/150] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:11:12 2022] PUT /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/users/ea009dc138d6487b9145db2c059d7a1f/roles/c54d8e4858914575b2ba91acf3233251 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:13.502089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a8978-68f7-4509-98ee-c4bf69004b10 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:13.571150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304e1b8-8919-41e1-b395-73fb359da0f9 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:13.571457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304e1b8-8919-41e1-b395-73fb359da0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:13.571730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304e1b8-8919-41e1-b395-73fb359da0f9 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:13.573505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 75/151] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:13.578912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08878bc-d8d7-4ad2-bf16-abd678787613 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:13.579181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08878bc-d8d7-4ad2-bf16-abd678787613 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:13.579456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08878bc-d8d7-4ad2-bf16-abd678787613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:13.601790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d08878bc-d8d7-4ad2-bf16-abd678787613 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:13.604861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d08878bc-d8d7-4ad2-bf16-abd678787613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:13.653994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a8978-68f7-4509-98ee-c4bf69004b10 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:13.677179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:13.708414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 76/152] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:13.716835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b64b8-29ab-43aa-a807-a9627a64e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:13.717796 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b64b8-29ab-43aa-a807-a9627a64e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:13.718082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b64b8-29ab-43aa-a807-a9627a64e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:13.718357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b64b8-29ab-43aa-a807-a9627a64e4a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:13.735316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b64b8-29ab-43aa-a807-a9627a64e4a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e59b64b8-29ab-43aa-a807-a9627a64e4a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:13.743588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:13.745006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:13.745754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:13.749142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 77/153] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:13.751985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08878bc-d8d7-4ad2-bf16-abd678787613 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:13.844738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:13.895614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08878bc-d8d7-4ad2-bf16-abd678787613 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:13.939404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64de5d-c644-4f90-b909-261a0c19192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:13.940021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64de5d-c644-4f90-b909-261a0c19192a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:13.940300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64de5d-c644-4f90-b909-261a0c19192a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:13.940572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64de5d-c644-4f90-b909-261a0c19192a None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:13.941341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64de5d-c644-4f90-b909-261a0c19192a None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:13.941642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64de5d-c644-4f90-b909-261a0c19192a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:13.943044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64de5d-c644-4f90-b909-261a0c19192a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:13.955415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 77/154] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:13.961006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b64b8-29ab-43aa-a807-a9627a64e4a4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:14.098251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b64b8-29ab-43aa-a807-a9627a64e4a4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:14.146643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e68c8-85c0-4845-aa67-3546a5b1fdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:14.147282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e68c8-85c0-4845-aa67-3546a5b1fdcd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:14.147646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e68c8-85c0-4845-aa67-3546a5b1fdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:14.147926 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e68c8-85c0-4845-aa67-3546a5b1fdcd None admin] PATH_INFO: `/v3/users/417da11974af471a8afca3fc24034882` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:14.153588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e68c8-85c0-4845-aa67-3546a5b1fdcd None admin] RBAC: Authorizing `identity:get_user(user_id=417da11974af471a8afca3fc24034882)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:14.155077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e68c8-85c0-4845-aa67-3546a5b1fdcd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:14.156717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 78/155] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:13 2022] GET /identity/v3/users/417da11974af471a8afca3fc24034882 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:14.162319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64de5d-c644-4f90-b909-261a0c19192a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:14.274222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e68c8-85c0-4845-aa67-3546a5b1fdcd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:14.287847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64de5d-c644-4f90-b909-261a0c19192a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:14.331558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:14.351576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d6e7c-2d71-46dd-8aff-353580e47d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:14.352120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d6e7c-2d71-46dd-8aff-353580e47d85 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:14.352404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d6e7c-2d71-46dd-8aff-353580e47d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:14.352677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d6e7c-2d71-46dd-8aff-353580e47d85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:14.389146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:14.480369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d6e7c-2d71-46dd-8aff-353580e47d85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:14.482304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d6e7c-2d71-46dd-8aff-353580e47d85 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:14.508052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e68c8-85c0-4845-aa67-3546a5b1fdcd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:14.599662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539dd0f-4e22-43d2-8fab-3a5d76917b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:14.600322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539dd0f-4e22-43d2-8fab-3a5d76917b82 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:14.600593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539dd0f-4e22-43d2-8fab-3a5d76917b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:14.600877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539dd0f-4e22-43d2-8fab-3a5d76917b82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:14.604987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 78/156] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:14.607178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3ac41-d882-4b67-9e96-4e642123e7ed None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:14.607650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3ac41-d882-4b67-9e96-4e642123e7ed None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:14.608209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3ac41-d882-4b67-9e96-4e642123e7ed None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:14.609752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 79/157] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:14.618365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb9a7-8c1d-4830-9ffa-730b8c10761d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:14.618870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb9a7-8c1d-4830-9ffa-730b8c10761d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:14.620833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb9a7-8c1d-4830-9ffa-730b8c10761d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:14.625612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539dd0f-4e22-43d2-8fab-3a5d76917b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-b539dd0f-4e22-43d2-8fab-3a5d76917b82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:14.630354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:14.631255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:14.632005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 79/158] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:11:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:14.632613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539dd0f-4e22-43d2-8fab-3a5d76917b82 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:14.663078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-344fb9a7-8c1d-4830-9ffa-730b8c10761d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:14.664263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-344fb9a7-8c1d-4830-9ffa-730b8c10761d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:14.709187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:14.792031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 80/159] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:14.792925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb9a7-8c1d-4830-9ffa-730b8c10761d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:14.826206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539dd0f-4e22-43d2-8fab-3a5d76917b82 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:14.829608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:14.934143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7df69-d8d8-4b4e-bbda-a5f86745f741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:14.934787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7df69-d8d8-4b4e-bbda-a5f86745f741 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:14.935062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7df69-d8d8-4b4e-bbda-a5f86745f741 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:14.935332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7df69-d8d8-4b4e-bbda-a5f86745f741 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:14.936480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7df69-d8d8-4b4e-bbda-a5f86745f741 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:14.937977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7df69-d8d8-4b4e-bbda-a5f86745f741 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:14.939613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 80/160] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:14 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:14.940740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100be6-e2f4-428b-bc67-5e0a2595d313 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:14.941057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100be6-e2f4-428b-bc67-5e0a2595d313 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:14.941326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100be6-e2f4-428b-bc67-5e0a2595d313 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:14.942513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 81/161] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:14.948885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4af821-eb6e-4592-9525-53756799cb72 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:14.948885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4af821-eb6e-4592-9525-53756799cb72 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:14.949215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4af821-eb6e-4592-9525-53756799cb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:14.977748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb9a7-8c1d-4830-9ffa-730b8c10761d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:14.980296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0b4af821-eb6e-4592-9525-53756799cb72 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:14.981791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0b4af821-eb6e-4592-9525-53756799cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:15.030100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bd395-1d5b-41e8-84d7-a19230ee808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:15.030708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bd395-1d5b-41e8-84d7-a19230ee808c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:15.030984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bd395-1d5b-41e8-84d7-a19230ee808c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:15.031249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800bd395-1d5b-41e8-84d7-a19230ee808c None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:15.031906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bd395-1d5b-41e8-84d7-a19230ee808c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:15.032201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bd395-1d5b-41e8-84d7-a19230ee808c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:15.033984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800bd395-1d5b-41e8-84d7-a19230ee808c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:15.052349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 81/162] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:15.053303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bd395-1d5b-41e8-84d7-a19230ee808c None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:15.081826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:15.137993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 82/163] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:15.139327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4af821-eb6e-4592-9525-53756799cb72 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:15.255519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800bd395-1d5b-41e8-84d7-a19230ee808c None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:15.310747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb5c4a-c83a-4374-9b14-22e8a0ca4cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:15.311398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb5c4a-c83a-4374-9b14-22e8a0ca4cad None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:15.311628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb5c4a-c83a-4374-9b14-22e8a0ca4cad None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:15.311866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb5c4a-c83a-4374-9b14-22e8a0ca4cad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:15.312586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb5c4a-c83a-4374-9b14-22e8a0ca4cad None admin] RBAC: Adding query filter params (role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c, user.id=417da11974af471a8afca3fc24034882) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:15.312920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb5c4a-c83a-4374-9b14-22e8a0ca4cad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:15.314949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb5c4a-c83a-4374-9b14-22e8a0ca4cad None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:15.327203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 82/164] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:15 2022] GET /identity/v3/role_assignments?user.id=417da11974af471a8afca3fc24034882&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:15.349810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4af821-eb6e-4592-9525-53756799cb72 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:15.353128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:15.402344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb5c4a-c83a-4374-9b14-22e8a0ca4cad None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:15.414546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3244a37-6e58-45ba-b42a-fa3cecba6d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:15.415190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3244a37-6e58-45ba-b42a-fa3cecba6d25 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:15.415462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3244a37-6e58-45ba-b42a-fa3cecba6d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:15.415728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3244a37-6e58-45ba-b42a-fa3cecba6d25 None admin] PATH_INFO: `/v3/users/4fac3b46e24449e193279323e9d64fe0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:15.417610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3244a37-6e58-45ba-b42a-fa3cecba6d25 None admin] RBAC: Authorizing `identity:get_user(user_id=4fac3b46e24449e193279323e9d64fe0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:15.418835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3244a37-6e58-45ba-b42a-fa3cecba6d25 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:15.420671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 83/165] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:15 2022] GET /identity/v3/users/4fac3b46e24449e193279323e9d64fe0 => generated 247 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:15.427775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3244a37-6e58-45ba-b42a-fa3cecba6d25 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:15.562211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:15.601799 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb5c4a-c83a-4374-9b14-22e8a0ca4cad None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:15.627379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3244a37-6e58-45ba-b42a-fa3cecba6d25 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:15.675789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae95b6-549c-40e1-bb06-6918138e231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:15.676431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae95b6-549c-40e1-bb06-6918138e231e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:15.676703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae95b6-549c-40e1-bb06-6918138e231e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:15.677111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae95b6-549c-40e1-bb06-6918138e231e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:15.690502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae95b6-549c-40e1-bb06-6918138e231e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-afae95b6-549c-40e1-bb06-6918138e231e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:15.696420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:15.697631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:15.698421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 83/166] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:15.704823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae95b6-549c-40e1-bb06-6918138e231e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:15.715293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab5dd0-1e6e-4d16-8c7c-87364fea80ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:15.715929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab5dd0-1e6e-4d16-8c7c-87364fea80ff None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:15.716204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab5dd0-1e6e-4d16-8c7c-87364fea80ff None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:15.716471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab5dd0-1e6e-4d16-8c7c-87364fea80ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:15.795584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:15.843348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab5dd0-1e6e-4d16-8c7c-87364fea80ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:15.844823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab5dd0-1e6e-4d16-8c7c-87364fea80ff None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:15.930766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae95b6-549c-40e1-bb06-6918138e231e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:15.958561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 84/167] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:15.959436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab5dd0-1e6e-4d16-8c7c-87364fea80ff None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:15.997587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28754cf3-cf83-4ea7-8baa-429eebbe5afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:15.998293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28754cf3-cf83-4ea7-8baa-429eebbe5afd None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:15.998570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28754cf3-cf83-4ea7-8baa-429eebbe5afd None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:15.998834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28754cf3-cf83-4ea7-8baa-429eebbe5afd None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:15.999489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28754cf3-cf83-4ea7-8baa-429eebbe5afd None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:15.999781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28754cf3-cf83-4ea7-8baa-429eebbe5afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:16.001190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28754cf3-cf83-4ea7-8baa-429eebbe5afd None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:16.014313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:16.018576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 84/168] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:16.019406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28754cf3-cf83-4ea7-8baa-429eebbe5afd None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:16.037668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:16.071404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab5dd0-1e6e-4d16-8c7c-87364fea80ff None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:16.109820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5f6ff-eae0-4e88-bf15-96a487a109e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:16.110691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5f6ff-eae0-4e88-bf15-96a487a109e6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:16.111108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5f6ff-eae0-4e88-bf15-96a487a109e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:16.111512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5f6ff-eae0-4e88-bf15-96a487a109e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:16.122679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5f6ff-eae0-4e88-bf15-96a487a109e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-8fc5f6ff-eae0-4e88-bf15-96a487a109e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:16.127699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:16.128756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:16.129548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:16.131681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 85/169] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:16.132897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5f6ff-eae0-4e88-bf15-96a487a109e6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:16.243293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5f6ff-eae0-4e88-bf15-96a487a109e6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:16.278912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28754cf3-cf83-4ea7-8baa-429eebbe5afd None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:16.283291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f89fc-f77c-4c54-b4a0-41b79f0acd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:16.284159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f89fc-f77c-4c54-b4a0-41b79f0acd86 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:16.284646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f89fc-f77c-4c54-b4a0-41b79f0acd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:16.285117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f89fc-f77c-4c54-b4a0-41b79f0acd86 None admin] PATH_INFO: `/v3/users/417da11974af471a8afca3fc24034882` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:16.286523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f89fc-f77c-4c54-b4a0-41b79f0acd86 None admin] RBAC: Authorizing `identity:get_user(user_id=417da11974af471a8afca3fc24034882)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:16.288651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f89fc-f77c-4c54-b4a0-41b79f0acd86 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:16.290724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 86/170] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:16 2022] GET /identity/v3/users/417da11974af471a8afca3fc24034882 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:16.293308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:16.297584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f89fc-f77c-4c54-b4a0-41b79f0acd86 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:16.376697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d66b45-0e5e-4b69-a5e5-77b86a538787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:16.377386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d66b45-0e5e-4b69-a5e5-77b86a538787 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:16.377546 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d66b45-0e5e-4b69-a5e5-77b86a538787 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:16.377799 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d66b45-0e5e-4b69-a5e5-77b86a538787 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:16.379165 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d66b45-0e5e-4b69-a5e5-77b86a538787 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:16.380620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d66b45-0e5e-4b69-a5e5-77b86a538787 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:16.382553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 85/171] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:16 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:16.383323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d66b45-0e5e-4b69-a5e5-77b86a538787 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:16.435160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f89fc-f77c-4c54-b4a0-41b79f0acd86 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:16.489991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0145b-8c29-4aa4-9169-fde576224182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:16.490608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0145b-8c29-4aa4-9169-fde576224182 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:16.490880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0145b-8c29-4aa4-9169-fde576224182 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:16.491155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0145b-8c29-4aa4-9169-fde576224182 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:16.491819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0145b-8c29-4aa4-9169-fde576224182 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:16.492118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0145b-8c29-4aa4-9169-fde576224182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:16.494171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0145b-8c29-4aa4-9169-fde576224182 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:16.511379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 87/172] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:16.512722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7aa19-a7b9-4352-bb13-ae81be137ecb None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:16.513055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7aa19-a7b9-4352-bb13-ae81be137ecb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:16.513334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7aa19-a7b9-4352-bb13-ae81be137ecb None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:16.514635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 88/173] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:16.515343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7aa19-a7b9-4352-bb13-ae81be137ecb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:16.516507 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:16.554513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d66b45-0e5e-4b69-a5e5-77b86a538787 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:16.592670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14953438-1f6e-4fd0-8417-43f1501c0293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:16.593587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14953438-1f6e-4fd0-8417-43f1501c0293 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:16.593964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14953438-1f6e-4fd0-8417-43f1501c0293 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:16.594331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14953438-1f6e-4fd0-8417-43f1501c0293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:16.670723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14953438-1f6e-4fd0-8417-43f1501c0293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:16.672318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14953438-1f6e-4fd0-8417-43f1501c0293 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:16.749207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:16.756930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:16.797209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 86/174] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:16.798303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14953438-1f6e-4fd0-8417-43f1501c0293 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:16.865761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7aa19-a7b9-4352-bb13-ae81be137ecb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:16.904846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705fbea-661e-4313-9a08-e57a61830466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:16.905381 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705fbea-661e-4313-9a08-e57a61830466 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:16.905656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705fbea-661e-4313-9a08-e57a61830466 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:16.905927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705fbea-661e-4313-9a08-e57a61830466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:16.906650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705fbea-661e-4313-9a08-e57a61830466 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c, user.id=4fac3b46e24449e193279323e9d64fe0) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:16.906954 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705fbea-661e-4313-9a08-e57a61830466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:16.908384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705fbea-661e-4313-9a08-e57a61830466 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:16.914480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 89/175] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:16 2022] GET /identity/v3/role_assignments?user.id=4fac3b46e24449e193279323e9d64fe0&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 608 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:16.915690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b1c1a-9bc4-45f2-a3eb-23ed4085908d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:16.915978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b1c1a-9bc4-45f2-a3eb-23ed4085908d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:16.916825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b1c1a-9bc4-45f2-a3eb-23ed4085908d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:16.921334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14953438-1f6e-4fd0-8417-43f1501c0293 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:16.958108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-639b1c1a-9bc4-45f2-a3eb-23ed4085908d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:16.959315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-639b1c1a-9bc4-45f2-a3eb-23ed4085908d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:16.977162 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b7fe7-ecd3-4d20-82c8-e292291ffba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:16.977817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b7fe7-ecd3-4d20-82c8-e292291ffba1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:16.978092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b7fe7-ecd3-4d20-82c8-e292291ffba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:16.978360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b7fe7-ecd3-4d20-82c8-e292291ffba1 None admin] PATH_INFO: `/v3/users/417da11974af471a8afca3fc24034882` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:16.983711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b7fe7-ecd3-4d20-82c8-e292291ffba1 None admin] RBAC: Authorizing `identity:get_user(user_id=417da11974af471a8afca3fc24034882)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:16.985273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b7fe7-ecd3-4d20-82c8-e292291ffba1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:16.987028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 87/176] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:16 2022] GET /identity/v3/users/417da11974af471a8afca3fc24034882 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:16.987827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b7fe7-ecd3-4d20-82c8-e292291ffba1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:17.072650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 90/177] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:17.074117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990af99-5526-4d10-9962-d1a82c4d14e8 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.074448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990af99-5526-4d10-9962-d1a82c4d14e8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.074759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990af99-5526-4d10-9962-d1a82c4d14e8 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.076208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 91/178] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:17.077071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990af99-5526-4d10-9962-d1a82c4d14e8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:17.094983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:17.138301 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:17.192606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b7fe7-ecd3-4d20-82c8-e292291ffba1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:17.194478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990af99-5526-4d10-9962-d1a82c4d14e8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:17.236133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66705805-c11e-44a0-a149-e111741feb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:17.236765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66705805-c11e-44a0-a149-e111741feb7a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.237072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66705805-c11e-44a0-a149-e111741feb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.237409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66705805-c11e-44a0-a149-e111741feb7a None admin] PATH_INFO: `/v3/projects/2bd636be4e09401ca420cb21fd0e3a0f` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.238465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66705805-c11e-44a0-a149-e111741feb7a None admin] RBAC: Authorizing `identity:get_project(project_id=2bd636be4e09401ca420cb21fd0e3a0f)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:17.239862 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66705805-c11e-44a0-a149-e111741feb7a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:17.241993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398466e-f7b9-4775-9a92-6420bb6377d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:17.242127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 88/179] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:16 2022] GET /identity/v3/projects/2bd636be4e09401ca420cb21fd0e3a0f => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:17.242206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398466e-f7b9-4775-9a92-6420bb6377d7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.242206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398466e-f7b9-4775-9a92-6420bb6377d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.242457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398466e-f7b9-4775-9a92-6420bb6377d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.248653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36528d1c-fc30-458a-898d-114c86f0e0e3 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.248653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36528d1c-fc30-458a-898d-114c86f0e0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.249051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36528d1c-fc30-458a-898d-114c86f0e0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.267575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-36528d1c-fc30-458a-898d-114c86f0e0e3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:17.268738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-36528d1c-fc30-458a-898d-114c86f0e0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:17.321814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398466e-f7b9-4775-9a92-6420bb6377d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:17.323453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398466e-f7b9-4775-9a92-6420bb6377d7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:17.345214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 89/180] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:17.345214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e0aa-2cf9-4131-bbbd-842833a77e2b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.345214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e0aa-2cf9-4131-bbbd-842833a77e2b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.345214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374e0aa-2cf9-4131-bbbd-842833a77e2b None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.345214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 90/181] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:17.345214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33502-e604-41a6-b3e6-11836c72021b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.345214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33502-e604-41a6-b3e6-11836c72021b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.345831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c33502-e604-41a6-b3e6-11836c72021b None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.345831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 91/182] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:17.345831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c33502-e604-41a6-b3e6-11836c72021b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:17.388231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:17.410251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:17.457006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 92/183] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:17.457006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da16145-741d-43d4-b44a-e05be54d4cef None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.457006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da16145-741d-43d4-b44a-e05be54d4cef None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.457006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da16145-741d-43d4-b44a-e05be54d4cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.463253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c33502-e604-41a6-b3e6-11836c72021b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:17.473824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6da16145-741d-43d4-b44a-e05be54d4cef None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:17.475011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6da16145-741d-43d4-b44a-e05be54d4cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:17.499996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e169dc-baad-40bb-8af0-2e3288e02099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:17.500885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e169dc-baad-40bb-8af0-2e3288e02099 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.501163 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e169dc-baad-40bb-8af0-2e3288e02099 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.501428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e169dc-baad-40bb-8af0-2e3288e02099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.502143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e169dc-baad-40bb-8af0-2e3288e02099 None admin] RBAC: Adding query filter params (role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=2bd636be4e09401ca420cb21fd0e3a0f, user.id=417da11974af471a8afca3fc24034882) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:17.502449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e169dc-baad-40bb-8af0-2e3288e02099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:17.503870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e169dc-baad-40bb-8af0-2e3288e02099 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:17.517198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 92/184] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:17 2022] GET /identity/v3/role_assignments?user.id=417da11974af471a8afca3fc24034882&scope.project.id=2bd636be4e09401ca420cb21fd0e3a0f&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:17.518815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc9991-a21d-4df4-a89f-ca6ef4a13ff4 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.519193 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc9991-a21d-4df4-a89f-ca6ef4a13ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.519564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc9991-a21d-4df4-a89f-ca6ef4a13ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.544851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d8bc9991-a21d-4df4-a89f-ca6ef4a13ff4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:17.546559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d8bc9991-a21d-4df4-a89f-ca6ef4a13ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:17.555015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 93/185] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:17.556324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da16145-741d-43d4-b44a-e05be54d4cef None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:17.644100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 93/186] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:17.650135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc9991-a21d-4df4-a89f-ca6ef4a13ff4 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:17.656878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:17.678876 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da16145-741d-43d4-b44a-e05be54d4cef None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:17.715971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:17.723603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a30ce-1386-4872-bac5-823b3888dd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:17.724452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a30ce-1386-4872-bac5-823b3888dd8b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.724899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a30ce-1386-4872-bac5-823b3888dd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.725332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a30ce-1386-4872-bac5-823b3888dd8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.737011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a30ce-1386-4872-bac5-823b3888dd8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5e3a30ce-1386-4872-bac5-823b3888dd8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:17.741729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:17.742649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:17.743434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 94/187] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:17.744155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a30ce-1386-4872-bac5-823b3888dd8b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:17.833190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc9991-a21d-4df4-a89f-ca6ef4a13ff4 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:17.885638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6db0d-2a6e-4c2b-82c3-fd6e96ea3011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:17.886276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6db0d-2a6e-4c2b-82c3-fd6e96ea3011 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.886553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6db0d-2a6e-4c2b-82c3-fd6e96ea3011 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.886819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6db0d-2a6e-4c2b-82c3-fd6e96ea3011 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.887980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6db0d-2a6e-4c2b-82c3-fd6e96ea3011 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:17.889456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6db0d-2a6e-4c2b-82c3-fd6e96ea3011 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:17.891245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 94/188] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:17 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:17.891843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6db0d-2a6e-4c2b-82c3-fd6e96ea3011 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:17.902701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a30ce-1386-4872-bac5-823b3888dd8b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:17.941901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4acd1-8a3d-4fea-942a-6d53f9341969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:17.942530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4acd1-8a3d-4fea-942a-6d53f9341969 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:17.942817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4acd1-8a3d-4fea-942a-6d53f9341969 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:17.943071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4acd1-8a3d-4fea-942a-6d53f9341969 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:17.944179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4acd1-8a3d-4fea-942a-6d53f9341969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:17.945668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4acd1-8a3d-4fea-942a-6d53f9341969 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:17.947434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 95/189] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:17.948081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4acd1-8a3d-4fea-942a-6d53f9341969 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:17.962453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:17.995671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:18.001361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6db0d-2a6e-4c2b-82c3-fd6e96ea3011 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.035535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb83143-e08e-47b8-9b26-110d3190030c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.036125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb83143-e08e-47b8-9b26-110d3190030c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.036395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb83143-e08e-47b8-9b26-110d3190030c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.036659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb83143-e08e-47b8-9b26-110d3190030c None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.037417 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb83143-e08e-47b8-9b26-110d3190030c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:18.037970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb83143-e08e-47b8-9b26-110d3190030c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:18.039443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb83143-e08e-47b8-9b26-110d3190030c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:18.047114 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4acd1-8a3d-4fea-942a-6d53f9341969 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.048047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 95/190] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:18.048844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb83143-e08e-47b8-9b26-110d3190030c None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:18.081692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeba73c-e237-41ca-a362-4ec66af15a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.082313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeba73c-e237-41ca-a362-4ec66af15a3e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.082580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeba73c-e237-41ca-a362-4ec66af15a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.082846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeba73c-e237-41ca-a362-4ec66af15a3e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.092987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeba73c-e237-41ca-a362-4ec66af15a3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-baeba73c-e237-41ca-a362-4ec66af15a3e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:18.097626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:18.098485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 96/191] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:18.099460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeba73c-e237-41ca-a362-4ec66af15a3e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:18.154214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb83143-e08e-47b8-9b26-110d3190030c None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.168410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:18.188293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754458fb-7c50-48cb-8d39-e65309dfca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.188918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754458fb-7c50-48cb-8d39-e65309dfca56 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.189193 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754458fb-7c50-48cb-8d39-e65309dfca56 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.189458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754458fb-7c50-48cb-8d39-e65309dfca56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.190166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754458fb-7c50-48cb-8d39-e65309dfca56 None admin] RBAC: Adding query filter params (role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c, user.id=417da11974af471a8afca3fc24034882) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:18.190464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754458fb-7c50-48cb-8d39-e65309dfca56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:18.191958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754458fb-7c50-48cb-8d39-e65309dfca56 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:18.197764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeba73c-e237-41ca-a362-4ec66af15a3e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.197928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 96/192] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:18 2022] GET /identity/v3/role_assignments?user.id=417da11974af471a8afca3fc24034882&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:18.198652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754458fb-7c50-48cb-8d39-e65309dfca56 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:18.227382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:18.250450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242edc6-0f7a-4566-b926-4d3d151fe819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.250996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242edc6-0f7a-4566-b926-4d3d151fe819 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.251274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242edc6-0f7a-4566-b926-4d3d151fe819 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.251547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242edc6-0f7a-4566-b926-4d3d151fe819 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.252219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242edc6-0f7a-4566-b926-4d3d151fe819 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:18.253787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242edc6-0f7a-4566-b926-4d3d151fe819 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:18.313354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754458fb-7c50-48cb-8d39-e65309dfca56 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.348455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816b5fa-a3b2-4464-ac41-1270138858c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.349109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816b5fa-a3b2-4464-ac41-1270138858c5 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.349383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816b5fa-a3b2-4464-ac41-1270138858c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.349646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816b5fa-a3b2-4464-ac41-1270138858c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.361620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816b5fa-a3b2-4464-ac41-1270138858c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8816b5fa-a3b2-4464-ac41-1270138858c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:18.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:18.367837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:18.368617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 97/193] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:18.369539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816b5fa-a3b2-4464-ac41-1270138858c5 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:18.381776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 97/194] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:18 2022] POST /identity/v3/users => generated 297 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:18.382614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242edc6-0f7a-4566-b926-4d3d151fe819 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:18.476142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:18.496077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816b5fa-a3b2-4464-ac41-1270138858c5 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.532331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898123c7-f58e-4893-9811-0f8421109790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.533288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898123c7-f58e-4893-9811-0f8421109790 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.533739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898123c7-f58e-4893-9811-0f8421109790 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.534240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898123c7-f58e-4893-9811-0f8421109790 None admin] PATH_INFO: `/v3/users/61705f82de4d4a0fb0b19ec86ea833e5` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.536510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242edc6-0f7a-4566-b926-4d3d151fe819 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.553809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898123c7-f58e-4893-9811-0f8421109790 None admin] RBAC: Authorizing `identity:get_user(user_id=61705f82de4d4a0fb0b19ec86ea833e5)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:18.555290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898123c7-f58e-4893-9811-0f8421109790 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:18.557234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 98/195] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:18 2022] GET /identity/v3/users/61705f82de4d4a0fb0b19ec86ea833e5 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:18.557873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898123c7-f58e-4893-9811-0f8421109790 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:18.568487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:18.579292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660da95-7d32-43b0-bd56-8f483bc0b579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.582474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660da95-7d32-43b0-bd56-8f483bc0b579 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.582474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660da95-7d32-43b0-bd56-8f483bc0b579 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.582474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660da95-7d32-43b0-bd56-8f483bc0b579 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.582474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660da95-7d32-43b0-bd56-8f483bc0b579 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:18.582474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660da95-7d32-43b0-bd56-8f483bc0b579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:18.582851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660da95-7d32-43b0-bd56-8f483bc0b579 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:18.592759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 98/196] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:18.596907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660da95-7d32-43b0-bd56-8f483bc0b579 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:18.687752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898123c7-f58e-4893-9811-0f8421109790 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.717145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660da95-7d32-43b0-bd56-8f483bc0b579 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.732765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bc221-193e-47a2-bae2-4c7130e3f16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.733412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bc221-193e-47a2-bae2-4c7130e3f16c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.733693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bc221-193e-47a2-bae2-4c7130e3f16c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.734036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bc221-193e-47a2-bae2-4c7130e3f16c None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.735069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bc221-193e-47a2-bae2-4c7130e3f16c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:18.735069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bc221-193e-47a2-bae2-4c7130e3f16c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:18.736560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bc221-193e-47a2-bae2-4c7130e3f16c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:18.747442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 99/197] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:18.748440 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bc221-193e-47a2-bae2-4c7130e3f16c None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:18.754622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:18.755646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55112d-de75-4262-88ad-23333bfa8294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.756319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55112d-de75-4262-88ad-23333bfa8294 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.756589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55112d-de75-4262-88ad-23333bfa8294 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.764846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55112d-de75-4262-88ad-23333bfa8294 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.828571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:18.839802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55112d-de75-4262-88ad-23333bfa8294 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:18.841525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55112d-de75-4262-88ad-23333bfa8294 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:18.888175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bc221-193e-47a2-bae2-4c7130e3f16c None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:18.923007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261a6f8-67ef-46ca-a84f-2a1edd8b7643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:18.923636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261a6f8-67ef-46ca-a84f-2a1edd8b7643 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:18.923910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261a6f8-67ef-46ca-a84f-2a1edd8b7643 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:18.924177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261a6f8-67ef-46ca-a84f-2a1edd8b7643 None admin] PATH_INFO: `/v3/users/ea009dc138d6487b9145db2c059d7a1f` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:18.925345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261a6f8-67ef-46ca-a84f-2a1edd8b7643 None admin] RBAC: Authorizing `identity:get_user(user_id=ea009dc138d6487b9145db2c059d7a1f)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:18.926792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261a6f8-67ef-46ca-a84f-2a1edd8b7643 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:18.928535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 100/198] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:18 2022] GET /identity/v3/users/ea009dc138d6487b9145db2c059d7a1f => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:18.929149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261a6f8-67ef-46ca-a84f-2a1edd8b7643 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:18.955311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 99/199] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:18.956406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55112d-de75-4262-88ad-23333bfa8294 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:19.033177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:19.037369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261a6f8-67ef-46ca-a84f-2a1edd8b7643 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:19.053130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:19.063612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55112d-de75-4262-88ad-23333bfa8294 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:19.075771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ce974-1877-4fdc-8a02-9750e63c9646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:19.076406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ce974-1877-4fdc-8a02-9750e63c9646 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:19.076682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ce974-1877-4fdc-8a02-9750e63c9646 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:19.076990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ce974-1877-4fdc-8a02-9750e63c9646 None admin] PATH_INFO: `/v3/users/61705f82de4d4a0fb0b19ec86ea833e5` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:19.078373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ce974-1877-4fdc-8a02-9750e63c9646 None admin] RBAC: Authorizing `identity:get_user(user_id=61705f82de4d4a0fb0b19ec86ea833e5)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:19.083170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ce974-1877-4fdc-8a02-9750e63c9646 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:19.083170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 101/200] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:18 2022] GET /identity/v3/users/61705f82de4d4a0fb0b19ec86ea833e5 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:19.083170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ce974-1877-4fdc-8a02-9750e63c9646 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:19.101568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952958d-0b05-4cdb-b229-8dcd3a854b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:19.102470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952958d-0b05-4cdb-b229-8dcd3a854b51 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:19.102744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952958d-0b05-4cdb-b229-8dcd3a854b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:19.103010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952958d-0b05-4cdb-b229-8dcd3a854b51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:19.189296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952958d-0b05-4cdb-b229-8dcd3a854b51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:19.190774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952958d-0b05-4cdb-b229-8dcd3a854b51 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:19.205432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ce974-1877-4fdc-8a02-9750e63c9646 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:19.254292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08690061-f8d9-451e-8e04-955c8fc8411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:19.255195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08690061-f8d9-451e-8e04-955c8fc8411b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:19.255477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08690061-f8d9-451e-8e04-955c8fc8411b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:19.255996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08690061-f8d9-451e-8e04-955c8fc8411b None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:19.257526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08690061-f8d9-451e-8e04-955c8fc8411b None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:19.258973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08690061-f8d9-451e-8e04-955c8fc8411b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:19.262460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 102/201] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:19 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:19.265495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08690061-f8d9-451e-8e04-955c8fc8411b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:19.289171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:19.301270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:19.317847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 100/202] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:19.319263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952958d-0b05-4cdb-b229-8dcd3a854b51 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:19.376141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08690061-f8d9-451e-8e04-955c8fc8411b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:19.416608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482ab19-ba78-4666-8702-6bc1e5041b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:19.417647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482ab19-ba78-4666-8702-6bc1e5041b97 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:19.417922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482ab19-ba78-4666-8702-6bc1e5041b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:19.418236 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482ab19-ba78-4666-8702-6bc1e5041b97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:19.453457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952958d-0b05-4cdb-b229-8dcd3a854b51 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:19.499508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482ab19-ba78-4666-8702-6bc1e5041b97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:19.501915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482ab19-ba78-4666-8702-6bc1e5041b97 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:19.502793 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c43faf-140e-46e5-88cd-67a8d2c7fecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:19.503492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c43faf-140e-46e5-88cd-67a8d2c7fecb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:19.503901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c43faf-140e-46e5-88cd-67a8d2c7fecb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:19.504494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c43faf-140e-46e5-88cd-67a8d2c7fecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:19.506235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c43faf-140e-46e5-88cd-67a8d2c7fecb None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=931647de90cc4037a03cac17bd65f53d, user.id=61705f82de4d4a0fb0b19ec86ea833e5) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:19.506651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c43faf-140e-46e5-88cd-67a8d2c7fecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:19.508163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c43faf-140e-46e5-88cd-67a8d2c7fecb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:19.512669 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:19.516421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 101/203] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:19 2022] GET /identity/v3/role_assignments?user.id=61705f82de4d4a0fb0b19ec86ea833e5&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:19.517226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c43faf-140e-46e5-88cd-67a8d2c7fecb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:19.592283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:19.649964 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c43faf-140e-46e5-88cd-67a8d2c7fecb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:19.696121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a43f28-267e-43f1-9cea-f5a8ddbec10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:19.696772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a43f28-267e-43f1-9cea-f5a8ddbec10b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:19.697068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a43f28-267e-43f1-9cea-f5a8ddbec10b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:19.698023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a43f28-267e-43f1-9cea-f5a8ddbec10b None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:19.703622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a43f28-267e-43f1-9cea-f5a8ddbec10b None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:19.705133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a43f28-267e-43f1-9cea-f5a8ddbec10b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:19.706883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 102/204] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:19 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:19.713589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a43f28-267e-43f1-9cea-f5a8ddbec10b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:19.734225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 103/205] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:19.740309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482ab19-ba78-4666-8702-6bc1e5041b97 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:19.872815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482ab19-ba78-4666-8702-6bc1e5041b97 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:19.883338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:19.914340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a43f28-267e-43f1-9cea-f5a8ddbec10b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:19.919710 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81dfde1-18ff-4027-b06a-56f06a0ab253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:19.920605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dfde1-18ff-4027-b06a-56f06a0ab253 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:19.921186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dfde1-18ff-4027-b06a-56f06a0ab253 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:19.922548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81dfde1-18ff-4027-b06a-56f06a0ab253 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:19.925290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81dfde1-18ff-4027-b06a-56f06a0ab253 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:19.930711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81dfde1-18ff-4027-b06a-56f06a0ab253 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:19.939495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 104/206] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:19 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:19.946121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81dfde1-18ff-4027-b06a-56f06a0ab253 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:19.980013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:20.026146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53fb4-a86a-45e1-a58e-4c73df21a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:20.026787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53fb4-a86a-45e1-a58e-4c73df21a255 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:20.033200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53fb4-a86a-45e1-a58e-4c73df21a255 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:20.033487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53fb4-a86a-45e1-a58e-4c73df21a255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:20.034256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d53fb4-a86a-45e1-a58e-4c73df21a255 None admin] RBAC: Adding query filter params (role.id=c54d8e4858914575b2ba91acf3233251, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c, user.id=ea009dc138d6487b9145db2c059d7a1f) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:20.034572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d53fb4-a86a-45e1-a58e-4c73df21a255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:20.036045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d53fb4-a86a-45e1-a58e-4c73df21a255 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:20.051021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 103/207] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:19 2022] GET /identity/v3/role_assignments?user.id=ea009dc138d6487b9145db2c059d7a1f&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=c54d8e4858914575b2ba91acf3233251 => generated 608 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:20.068454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81dfde1-18ff-4027-b06a-56f06a0ab253 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:20.112282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ecd45-534a-4ac1-8650-a854aa5a81bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:20.113312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ecd45-534a-4ac1-8650-a854aa5a81bf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:20.113785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ecd45-534a-4ac1-8650-a854aa5a81bf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:20.114245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ecd45-534a-4ac1-8650-a854aa5a81bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:20.115218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ecd45-534a-4ac1-8650-a854aa5a81bf None admin] RBAC: Adding query filter params (role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=931647de90cc4037a03cac17bd65f53d, user.id=61705f82de4d4a0fb0b19ec86ea833e5) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:20.115804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ecd45-534a-4ac1-8650-a854aa5a81bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:20.117720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ecd45-534a-4ac1-8650-a854aa5a81bf None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:20.125460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 105/208] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:19 2022] GET /identity/v3/role_assignments?user.id=61705f82de4d4a0fb0b19ec86ea833e5&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:20.610361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca965fc-814b-4d16-bdce-550db2544791 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:20.610894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca965fc-814b-4d16-bdce-550db2544791 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:20.610894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca965fc-814b-4d16-bdce-550db2544791 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:20.612234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 104/209] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:20.618032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc48b32-96c3-4458-a044-e39a7e2154b0 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:20.618473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc48b32-96c3-4458-a044-e39a7e2154b0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:20.618576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc48b32-96c3-4458-a044-e39a7e2154b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:20.642006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6cc48b32-96c3-4458-a044-e39a7e2154b0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:20.643186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6cc48b32-96c3-4458-a044-e39a7e2154b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:20.686854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:20.710306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f64c2-e109-4a39-a393-02087ef02072 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:20.710934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f64c2-e109-4a39-a393-02087ef02072 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:20.711504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f64c2-e109-4a39-a393-02087ef02072 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:20.713152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 105/210] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:20.718652 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9e61a-2ed2-4a97-83c0-4279c89b72df None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:20.719165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9e61a-2ed2-4a97-83c0-4279c89b72df None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:20.719663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9e61a-2ed2-4a97-83c0-4279c89b72df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:20.730483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 106/211] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:20.739270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1bf9e61a-2ed2-4a97-83c0-4279c89b72df None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:20.740734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1bf9e61a-2ed2-4a97-83c0-4279c89b72df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:20.811010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 106/212] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:21.023010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc48b32-96c3-4458-a044-e39a7e2154b0 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:21.082985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9e61a-2ed2-4a97-83c0-4279c89b72df None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:21.136965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc48b32-96c3-4458-a044-e39a7e2154b0 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:21.187044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb325f-a33d-4d73-8aeb-f7e6d5d0d51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:21.188064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb325f-a33d-4d73-8aeb-f7e6d5d0d51f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:21.188337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb325f-a33d-4d73-8aeb-f7e6d5d0d51f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:21.188599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb325f-a33d-4d73-8aeb-f7e6d5d0d51f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:21.203397 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb325f-a33d-4d73-8aeb-f7e6d5d0d51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-cabb325f-a33d-4d73-8aeb-f7e6d5d0d51f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:21.210134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:21.210998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:21.211759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:21.214129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9e61a-2ed2-4a97-83c0-4279c89b72df None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:21.216111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 107/213] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:21.218740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb325f-a33d-4d73-8aeb-f7e6d5d0d51f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:21.234355 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:21.254651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5a168-1ffa-4078-90a3-22ec6e7a9b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:21.255284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5a168-1ffa-4078-90a3-22ec6e7a9b78 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:21.255562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5a168-1ffa-4078-90a3-22ec6e7a9b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:21.255896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b5a168-1ffa-4078-90a3-22ec6e7a9b78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:21.257001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:21.257113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5a168-1ffa-4078-90a3-22ec6e7a9b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:21.258475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b5a168-1ffa-4078-90a3-22ec6e7a9b78 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:21.261813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 107/214] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:21.267087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5a168-1ffa-4078-90a3-22ec6e7a9b78 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:21.345966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb325f-a33d-4d73-8aeb-f7e6d5d0d51f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:21.386920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98074636-a4f8-4900-b1fb-3d92d73bbf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:21.388280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98074636-a4f8-4900-b1fb-3d92d73bbf2f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:21.388987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98074636-a4f8-4900-b1fb-3d92d73bbf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:21.389449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98074636-a4f8-4900-b1fb-3d92d73bbf2f None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:21.390286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98074636-a4f8-4900-b1fb-3d92d73bbf2f None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:21.390771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98074636-a4f8-4900-b1fb-3d92d73bbf2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:21.392326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98074636-a4f8-4900-b1fb-3d92d73bbf2f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:21.398637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b5a168-1ffa-4078-90a3-22ec6e7a9b78 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:21.402017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 108/215] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:21.408931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98074636-a4f8-4900-b1fb-3d92d73bbf2f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:21.442280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130c17a-77d0-4954-a778-5ed17459ca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:21.442894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c17a-77d0-4954-a778-5ed17459ca16 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:21.443172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c17a-77d0-4954-a778-5ed17459ca16 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:21.443442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c17a-77d0-4954-a778-5ed17459ca16 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:21.444098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130c17a-77d0-4954-a778-5ed17459ca16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:21.445563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130c17a-77d0-4954-a778-5ed17459ca16 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:21.509267 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:21.519228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98074636-a4f8-4900-b1fb-3d92d73bbf2f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:21.552701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b76c5b-09a1-415c-808a-552f84922f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:21.553348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b76c5b-09a1-415c-808a-552f84922f8f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:21.553622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b76c5b-09a1-415c-808a-552f84922f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:21.553883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b76c5b-09a1-415c-808a-552f84922f8f None admin] PATH_INFO: `/v3/users/87d9337ae63047969a1c87e508676a08` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:21.569082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b76c5b-09a1-415c-808a-552f84922f8f None admin] RBAC: Authorizing `identity:get_user(user_id=87d9337ae63047969a1c87e508676a08)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:21.570736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b76c5b-09a1-415c-808a-552f84922f8f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:21.572983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 109/216] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:21 2022] GET /identity/v3/users/87d9337ae63047969a1c87e508676a08 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:21.579387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b76c5b-09a1-415c-808a-552f84922f8f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:21.584955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 108/217] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:21 2022] POST /identity/v3/users => generated 297 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:11:21.700487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ce4c7-789c-4e80-a9c4-077de3d125ff None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:21.701126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ce4c7-789c-4e80-a9c4-077de3d125ff None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:21.701126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ce4c7-789c-4e80-a9c4-077de3d125ff None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:21.702839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 109/218] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:21.708285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b59617-c9ce-4b5f-9080-da38c8ff2299 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:21.708593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b59617-c9ce-4b5f-9080-da38c8ff2299 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:21.708869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b59617-c9ce-4b5f-9080-da38c8ff2299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:21.712591 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b76c5b-09a1-415c-808a-552f84922f8f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:21.731999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-08b59617-c9ce-4b5f-9080-da38c8ff2299 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:21.733753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-08b59617-c9ce-4b5f-9080-da38c8ff2299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:21.757773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113cfc3-2326-4d62-b122-67c03160d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:21.758475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113cfc3-2326-4d62-b122-67c03160d1fd None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:21.758818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113cfc3-2326-4d62-b122-67c03160d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:21.759916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113cfc3-2326-4d62-b122-67c03160d1fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:21.784073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:21.785859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:21.830464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 110/219] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:21.855238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113cfc3-2326-4d62-b122-67c03160d1fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:21.856969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113cfc3-2326-4d62-b122-67c03160d1fd None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:21.965721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 110/220] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:11:21.972702 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b59617-c9ce-4b5f-9080-da38c8ff2299 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:22.089380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b59617-c9ce-4b5f-9080-da38c8ff2299 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:22.125225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203db04d-b9ce-4097-9856-9c7f21b1cf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:22.125868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203db04d-b9ce-4097-9856-9c7f21b1cf9c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:22.126187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203db04d-b9ce-4097-9856-9c7f21b1cf9c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:22.126469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203db04d-b9ce-4097-9856-9c7f21b1cf9c None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:22.127603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203db04d-b9ce-4097-9856-9c7f21b1cf9c None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:22.129159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203db04d-b9ce-4097-9856-9c7f21b1cf9c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:22.130791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 111/221] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:21 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:22.137144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113cfc3-2326-4d62-b122-67c03160d1fd None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:22.154274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203db04d-b9ce-4097-9856-9c7f21b1cf9c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:22.202617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:22.221563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:22.276740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203db04d-b9ce-4097-9856-9c7f21b1cf9c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:22.304912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113cfc3-2326-4d62-b122-67c03160d1fd None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:22.325580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88407ea8-0a8a-446b-8c8e-436293b2f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:22.326224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88407ea8-0a8a-446b-8c8e-436293b2f54d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:22.326566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88407ea8-0a8a-446b-8c8e-436293b2f54d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:22.326914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88407ea8-0a8a-446b-8c8e-436293b2f54d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:22.341296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88407ea8-0a8a-446b-8c8e-436293b2f54d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:22.342253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8567baa-1925-4159-b46a-5c9c3291f1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:22.342921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8567baa-1925-4159-b46a-5c9c3291f1ae None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:22.343197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8567baa-1925-4159-b46a-5c9c3291f1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:22.343465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8567baa-1925-4159-b46a-5c9c3291f1ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:22.344188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8567baa-1925-4159-b46a-5c9c3291f1ae None admin] RBAC: Adding query filter params (role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=931647de90cc4037a03cac17bd65f53d, user.id=87d9337ae63047969a1c87e508676a08) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:22.344491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8567baa-1925-4159-b46a-5c9c3291f1ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:22.345945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8567baa-1925-4159-b46a-5c9c3291f1ae None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-88407ea8-0a8a-446b-8c8e-436293b2f54d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:22.346051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:22.346827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:22.348014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 112/222] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:11:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:11:22.349000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62faf7a-0290-4332-a668-2346a1bd8cff None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:22.351443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62faf7a-0290-4332-a668-2346a1bd8cff None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:22.351443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62faf7a-0290-4332-a668-2346a1bd8cff None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:22.351443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 113/223] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:22.353098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62faf7a-0290-4332-a668-2346a1bd8cff None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:22.357855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 111/224] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:22 2022] GET /identity/v3/role_assignments?user.id=87d9337ae63047969a1c87e508676a08&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:22.359085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058a104-0c74-47d2-b417-d80a2d3f9263 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:22.359355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058a104-0c74-47d2-b417-d80a2d3f9263 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:22.359625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058a104-0c74-47d2-b417-d80a2d3f9263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:22.385062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-7058a104-0c74-47d2-b417-d80a2d3f9263 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:22.386355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7058a104-0c74-47d2-b417-d80a2d3f9263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:22.502014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 112/225] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:22.515247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:22.521136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62faf7a-0290-4332-a668-2346a1bd8cff None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:22.560377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c00da3-231d-431c-be1e-0850edb6681d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:22.561033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c00da3-231d-431c-be1e-0850edb6681d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:22.561312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c00da3-231d-431c-be1e-0850edb6681d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:22.561588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c00da3-231d-431c-be1e-0850edb6681d None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:22.562263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c00da3-231d-431c-be1e-0850edb6681d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:22.562618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c00da3-231d-431c-be1e-0850edb6681d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:22.564003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c00da3-231d-431c-be1e-0850edb6681d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:22.573816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 114/226] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:22.579018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058a104-0c74-47d2-b417-d80a2d3f9263 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:22.663079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:22.733777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058a104-0c74-47d2-b417-d80a2d3f9263 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:22.768141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616ebf4-f803-4da4-924f-fea37709e2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:22.768689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616ebf4-f803-4da4-924f-fea37709e2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:22.769005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616ebf4-f803-4da4-924f-fea37709e2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:22.769276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616ebf4-f803-4da4-924f-fea37709e2f1 None admin] PATH_INFO: `/v3/users/61705f82de4d4a0fb0b19ec86ea833e5` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:22.770412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616ebf4-f803-4da4-924f-fea37709e2f1 None admin] RBAC: Authorizing `identity:get_user(user_id=61705f82de4d4a0fb0b19ec86ea833e5)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:22.771852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616ebf4-f803-4da4-924f-fea37709e2f1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:22.773762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 113/227] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:22 2022] GET /identity/v3/users/61705f82de4d4a0fb0b19ec86ea833e5 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:22.779582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c00da3-231d-431c-be1e-0850edb6681d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:22.821528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616ebf4-f803-4da4-924f-fea37709e2f1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:22.920181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c00da3-231d-431c-be1e-0850edb6681d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:22.977181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b96b3-7b2a-4e4c-889b-95a7d30135de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:22.978214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b96b3-7b2a-4e4c-889b-95a7d30135de None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:22.980868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b96b3-7b2a-4e4c-889b-95a7d30135de None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:22.981605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b96b3-7b2a-4e4c-889b-95a7d30135de None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:22.984084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b96b3-7b2a-4e4c-889b-95a7d30135de None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:22.987217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b96b3-7b2a-4e4c-889b-95a7d30135de None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:22.990059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 115/228] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:22 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:22.997919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b96b3-7b2a-4e4c-889b-95a7d30135de None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:23.027131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616ebf4-f803-4da4-924f-fea37709e2f1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:23.051301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:23.057306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:23.103650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be71e1-81af-4e0d-b95a-81e74b3c9d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:23.104303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be71e1-81af-4e0d-b95a-81e74b3c9d7b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:23.104575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be71e1-81af-4e0d-b95a-81e74b3c9d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:23.104855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be71e1-81af-4e0d-b95a-81e74b3c9d7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:23.123645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be71e1-81af-4e0d-b95a-81e74b3c9d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-e2be71e1-81af-4e0d-b95a-81e74b3c9d7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:23.131146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:23.132027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:23.132954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:23.140033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 114/229] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:23.142663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be71e1-81af-4e0d-b95a-81e74b3c9d7b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:23.204692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b96b3-7b2a-4e4c-889b-95a7d30135de None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:23.276610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7668c-23aa-47cc-a44c-c2496e276750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:23.277377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7668c-23aa-47cc-a44c-c2496e276750 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:23.277596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7668c-23aa-47cc-a44c-c2496e276750 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:23.277872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7668c-23aa-47cc-a44c-c2496e276750 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d/users/61705f82de4d4a0fb0b19ec86ea833e5/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:23.282385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be71e1-81af-4e0d-b95a-81e74b3c9d7b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:23.282584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7668c-23aa-47cc-a44c-c2496e276750 None admin] RBAC: Authorizing `identity:create_grant(project_id=931647de90cc4037a03cac17bd65f53d, user_id=61705f82de4d4a0fb0b19ec86ea833e5, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:23.284019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7668c-23aa-47cc-a44c-c2496e276750 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:23.321069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829dcd5-a835-480d-8909-f23c4e711d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:23.321647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829dcd5-a835-480d-8909-f23c4e711d4f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:23.321918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829dcd5-a835-480d-8909-f23c4e711d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:23.322295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829dcd5-a835-480d-8909-f23c4e711d4f None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:23.323043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829dcd5-a835-480d-8909-f23c4e711d4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:23.323337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829dcd5-a835-480d-8909-f23c4e711d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:23.324714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829dcd5-a835-480d-8909-f23c4e711d4f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:23.333309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 115/230] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:23.338987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829dcd5-a835-480d-8909-f23c4e711d4f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:23.372350 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:23.392756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 116/231] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:11:22 2022] PUT /identity/v3/projects/931647de90cc4037a03cac17bd65f53d/users/61705f82de4d4a0fb0b19ec86ea833e5/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:23.452365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829dcd5-a835-480d-8909-f23c4e711d4f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:23.488480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad2d02-f66a-4ed5-897c-b57fd1a7cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:23.489162 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad2d02-f66a-4ed5-897c-b57fd1a7cb44 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:23.489437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad2d02-f66a-4ed5-897c-b57fd1a7cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:23.489717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad2d02-f66a-4ed5-897c-b57fd1a7cb44 None admin] PATH_INFO: `/v3/users/61705f82de4d4a0fb0b19ec86ea833e5` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:23.490861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad2d02-f66a-4ed5-897c-b57fd1a7cb44 None admin] RBAC: Authorizing `identity:get_user(user_id=61705f82de4d4a0fb0b19ec86ea833e5)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:23.492318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ad2d02-f66a-4ed5-897c-b57fd1a7cb44 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:23.494143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 116/232] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:23 2022] GET /identity/v3/users/61705f82de4d4a0fb0b19ec86ea833e5 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:23.499931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7668c-23aa-47cc-a44c-c2496e276750 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:23.642289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:23.655658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7668c-23aa-47cc-a44c-c2496e276750 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:23.692415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25d4dc-63eb-4f72-8cd8-b57c705a5a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:23.699808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25d4dc-63eb-4f72-8cd8-b57c705a5a58 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:23.699808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25d4dc-63eb-4f72-8cd8-b57c705a5a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:23.699808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25d4dc-63eb-4f72-8cd8-b57c705a5a58 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:23.699808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25d4dc-63eb-4f72-8cd8-b57c705a5a58 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:23.699808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25d4dc-63eb-4f72-8cd8-b57c705a5a58 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:23.699808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 117/233] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:23 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:23.704713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad2d02-f66a-4ed5-897c-b57fd1a7cb44 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:23.814138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad2d02-f66a-4ed5-897c-b57fd1a7cb44 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:23.816999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:23.851148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d37cb5-a226-47eb-af1c-d0e9e5080a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:23.851812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37cb5-a226-47eb-af1c-d0e9e5080a45 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:23.852258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37cb5-a226-47eb-af1c-d0e9e5080a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:23.852529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d37cb5-a226-47eb-af1c-d0e9e5080a45 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d/users/61705f82de4d4a0fb0b19ec86ea833e5/roles/1fbe1ab7a2864ba1818c7038baf57c34` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:23.855588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37cb5-a226-47eb-af1c-d0e9e5080a45 None admin] RBAC: Authorizing `identity:create_grant(project_id=931647de90cc4037a03cac17bd65f53d, user_id=61705f82de4d4a0fb0b19ec86ea833e5, role_id=1fbe1ab7a2864ba1818c7038baf57c34)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:23.857338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d37cb5-a226-47eb-af1c-d0e9e5080a45 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:24.033634 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 117/234] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:11:23 2022] PUT /identity/v3/projects/931647de90cc4037a03cac17bd65f53d/users/61705f82de4d4a0fb0b19ec86ea833e5/roles/1fbe1ab7a2864ba1818c7038baf57c34 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:11:24.239712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44017feb-3eec-4a78-b367-2168c76c5615 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:24.240226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44017feb-3eec-4a78-b367-2168c76c5615 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:24.240226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44017feb-3eec-4a78-b367-2168c76c5615 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:24.241915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 118/235] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:24.249371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c8904-ee36-478f-a6c6-1f5e5b8309e1 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:24.249669 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c8904-ee36-478f-a6c6-1f5e5b8309e1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:24.249921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c8904-ee36-478f-a6c6-1f5e5b8309e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:24.271476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d43c8904-ee36-478f-a6c6-1f5e5b8309e1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:24.272677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d43c8904-ee36-478f-a6c6-1f5e5b8309e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:24.300440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579c08e-c4c2-4d78-8a1a-fca639e21f2d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:24.300733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579c08e-c4c2-4d78-8a1a-fca639e21f2d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:24.300966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579c08e-c4c2-4d78-8a1a-fca639e21f2d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:24.302570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 119/236] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:11:24.309681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da8460-f8a7-4eff-a641-050dcd87d650 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:24.310089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da8460-f8a7-4eff-a641-050dcd87d650 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:24.310461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da8460-f8a7-4eff-a641-050dcd87d650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:24.331403 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-71da8460-f8a7-4eff-a641-050dcd87d650 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:24.332597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-71da8460-f8a7-4eff-a641-050dcd87d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:24.351341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 118/237] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:24.414472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 120/238] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:24.651988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12cce6-5c05-482c-9322-3e3ab12f05af None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:24.663012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12cce6-5c05-482c-9322-3e3ab12f05af None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:24.663310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12cce6-5c05-482c-9322-3e3ab12f05af None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:24.664924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 119/239] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:24 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:24.671038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c16048-2e0f-4e86-ba5f-af1dc672e695 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:24.671328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c16048-2e0f-4e86-ba5f-af1dc672e695 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:24.671603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c16048-2e0f-4e86-ba5f-af1dc672e695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:24.697841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-25c16048-2e0f-4e86-ba5f-af1dc672e695 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:24.699043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-25c16048-2e0f-4e86-ba5f-af1dc672e695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:24.773846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12cce6-5c05-482c-9322-3e3ab12f05af None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:24.807365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 121/240] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:24.808268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c16048-2e0f-4e86-ba5f-af1dc672e695 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:24.833362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:24.941043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:24.958454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12cce6-5c05-482c-9322-3e3ab12f05af None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:24.992194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c16048-2e0f-4e86-ba5f-af1dc672e695 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:25.019714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cd867-7a9e-4612-970d-d1a9da8c6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:25.020558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cd867-7a9e-4612-970d-d1a9da8c6c20 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.020860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cd867-7a9e-4612-970d-d1a9da8c6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.021133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cd867-7a9e-4612-970d-d1a9da8c6c20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.023347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cd867-7a9e-4612-970d-d1a9da8c6c20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:25.024828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cd867-7a9e-4612-970d-d1a9da8c6c20 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:25.027476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 120/241] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:25.033911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cd867-7a9e-4612-970d-d1a9da8c6c20 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:25.066766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ad12f-ab03-4b8e-8bd0-e43d89dcdc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:25.069229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ad12f-ab03-4b8e-8bd0-e43d89dcdc07 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.069648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ad12f-ab03-4b8e-8bd0-e43d89dcdc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.071941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ad12f-ab03-4b8e-8bd0-e43d89dcdc07 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.094129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ad12f-ab03-4b8e-8bd0-e43d89dcdc07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-0f6ad12f-ab03-4b8e-8bd0-e43d89dcdc07 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:25.100140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:25.101044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:25.101802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:25.105203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 122/242] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:25.111212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ad12f-ab03-4b8e-8bd0-e43d89dcdc07 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:25.197807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cd867-7a9e-4612-970d-d1a9da8c6c20 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:25.209333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:25.241184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ad12f-ab03-4b8e-8bd0-e43d89dcdc07 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:25.248092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907dc55-8f8b-4571-b672-b5084bdc1d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:25.248713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907dc55-8f8b-4571-b672-b5084bdc1d6b None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.249021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907dc55-8f8b-4571-b672-b5084bdc1d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.249364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907dc55-8f8b-4571-b672-b5084bdc1d6b None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.250012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907dc55-8f8b-4571-b672-b5084bdc1d6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:25.251452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907dc55-8f8b-4571-b672-b5084bdc1d6b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:25.281108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c5a98-4ce6-4bce-b14b-0c4e0a5ede76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:25.281825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c5a98-4ce6-4bce-b14b-0c4e0a5ede76 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.282150 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c5a98-4ce6-4bce-b14b-0c4e0a5ede76 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.282428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c5a98-4ce6-4bce-b14b-0c4e0a5ede76 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.283125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c5a98-4ce6-4bce-b14b-0c4e0a5ede76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:25.283422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c5a98-4ce6-4bce-b14b-0c4e0a5ede76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:25.284857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c5a98-4ce6-4bce-b14b-0c4e0a5ede76 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:25.294147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 123/243] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:25.294770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c5a98-4ce6-4bce-b14b-0c4e0a5ede76 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:25.346264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 121/244] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:25 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:11:25.347361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907dc55-8f8b-4571-b672-b5084bdc1d6b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:25.368166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:25.435733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c5a98-4ce6-4bce-b14b-0c4e0a5ede76 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:25.507646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49583a-84f6-4800-b6fe-46320a795cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:25.508193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49583a-84f6-4800-b6fe-46320a795cbb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.508479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49583a-84f6-4800-b6fe-46320a795cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.508759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49583a-84f6-4800-b6fe-46320a795cbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.530953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:25.536951 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907dc55-8f8b-4571-b672-b5084bdc1d6b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:25.541787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49583a-84f6-4800-b6fe-46320a795cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-7e49583a-84f6-4800-b6fe-46320a795cbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:25.548821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:25.550007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:25.550949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:25.561135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 124/245] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:25.562135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49583a-84f6-4800-b6fe-46320a795cbb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:25.613458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f9890-7ade-4b7b-ac53-4dff4a9faf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:25.614208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f9890-7ade-4b7b-ac53-4dff4a9faf6d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.614545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f9890-7ade-4b7b-ac53-4dff4a9faf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.615011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f9890-7ade-4b7b-ac53-4dff4a9faf6d None admin] PATH_INFO: `/v3/users/fc7d6b6a272e4b9e8c23e879e7ab47da` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.636991 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f9890-7ade-4b7b-ac53-4dff4a9faf6d None admin] RBAC: Authorizing `identity:get_user(user_id=fc7d6b6a272e4b9e8c23e879e7ab47da)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:25.638776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f9890-7ade-4b7b-ac53-4dff4a9faf6d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:25.642023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 122/246] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:25 2022] GET /identity/v3/users/fc7d6b6a272e4b9e8c23e879e7ab47da => generated 297 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:25.656211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f9890-7ade-4b7b-ac53-4dff4a9faf6d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:25.738401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49583a-84f6-4800-b6fe-46320a795cbb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:25.758908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:25.797822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf802cca-8fe8-40b5-84b5-07bb3df1741c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:25.798862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf802cca-8fe8-40b5-84b5-07bb3df1741c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.800339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf802cca-8fe8-40b5-84b5-07bb3df1741c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.800862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf802cca-8fe8-40b5-84b5-07bb3df1741c None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.802154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf802cca-8fe8-40b5-84b5-07bb3df1741c None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:25.802680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf802cca-8fe8-40b5-84b5-07bb3df1741c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:25.805445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf802cca-8fe8-40b5-84b5-07bb3df1741c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:25.818605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f9890-7ade-4b7b-ac53-4dff4a9faf6d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:25.827956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 125/247] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:25.834264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf802cca-8fe8-40b5-84b5-07bb3df1741c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:25.871293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278a763-4c32-47e4-874b-35fbdef60bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:25.873388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278a763-4c32-47e4-874b-35fbdef60bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:25.874346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278a763-4c32-47e4-874b-35fbdef60bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:25.874824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278a763-4c32-47e4-874b-35fbdef60bf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:25.977233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:25.980851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278a763-4c32-47e4-874b-35fbdef60bf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:25.982670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278a763-4c32-47e4-874b-35fbdef60bf7 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:26.021262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf802cca-8fe8-40b5-84b5-07bb3df1741c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:26.077426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe95ab3-3cf9-4839-8d1a-d07672077a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:26.078374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe95ab3-3cf9-4839-8d1a-d07672077a09 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:26.078646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe95ab3-3cf9-4839-8d1a-d07672077a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:26.078915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe95ab3-3cf9-4839-8d1a-d07672077a09 None admin] PATH_INFO: `/v3/users/87d9337ae63047969a1c87e508676a08` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:26.080718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe95ab3-3cf9-4839-8d1a-d07672077a09 None admin] RBAC: Authorizing `identity:get_user(user_id=87d9337ae63047969a1c87e508676a08)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:26.082570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe95ab3-3cf9-4839-8d1a-d07672077a09 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:26.084946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 126/248] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:25 2022] GET /identity/v3/users/87d9337ae63047969a1c87e508676a08 => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:26.089021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616f6a-8a80-4de3-84dc-67bb70c2fb75 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:26.089318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616f6a-8a80-4de3-84dc-67bb70c2fb75 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:26.089573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b616f6a-8a80-4de3-84dc-67bb70c2fb75 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:26.091348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 127/249] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:26.091711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:26.093598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b616f6a-8a80-4de3-84dc-67bb70c2fb75 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:26.139698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 123/250] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:26.141512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacb97e-f64b-48ad-889f-0875277b8d4c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:26.142602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacb97e-f64b-48ad-889f-0875277b8d4c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:26.143072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacb97e-f64b-48ad-889f-0875277b8d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:26.171194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-bdacb97e-f64b-48ad-889f-0875277b8d4c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:26.174414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-bdacb97e-f64b-48ad-889f-0875277b8d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:26.254951 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b616f6a-8a80-4de3-84dc-67bb70c2fb75 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:26.292388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 124/251] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:26.293815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacb97e-f64b-48ad-889f-0875277b8d4c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:26.296464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:26.317633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6d7c6-8b05-402d-b758-c233e2475a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:26.318426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6d7c6-8b05-402d-b758-c233e2475a10 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:26.318855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6d7c6-8b05-402d-b758-c233e2475a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:26.319429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6d7c6-8b05-402d-b758-c233e2475a10 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:26.321217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6d7c6-8b05-402d-b758-c233e2475a10 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:26.323577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6d7c6-8b05-402d-b758-c233e2475a10 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:26.331783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 128/252] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:26 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:26.333101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6d7c6-8b05-402d-b758-c233e2475a10 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:26.431961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacb97e-f64b-48ad-889f-0875277b8d4c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:26.477773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:26.490276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e196eb-816c-4c54-ada5-bbace363aa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:26.490925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e196eb-816c-4c54-ada5-bbace363aa23 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:26.491252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e196eb-816c-4c54-ada5-bbace363aa23 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:26.491516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e196eb-816c-4c54-ada5-bbace363aa23 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:26.494848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e196eb-816c-4c54-ada5-bbace363aa23 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:26.496398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e196eb-816c-4c54-ada5-bbace363aa23 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:26.498421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 125/253] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:26 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:26.504709 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e196eb-816c-4c54-ada5-bbace363aa23 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:26.558377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6d7c6-8b05-402d-b758-c233e2475a10 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:26.619823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b81c1-354c-4fc5-8291-5ad86c073f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:26.621097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b81c1-354c-4fc5-8291-5ad86c073f70 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:26.622396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b81c1-354c-4fc5-8291-5ad86c073f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:26.622875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b81c1-354c-4fc5-8291-5ad86c073f70 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d/users/87d9337ae63047969a1c87e508676a08/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:26.629152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b81c1-354c-4fc5-8291-5ad86c073f70 None admin] RBAC: Authorizing `identity:create_grant(project_id=931647de90cc4037a03cac17bd65f53d, user_id=87d9337ae63047969a1c87e508676a08, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:26.630823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b81c1-354c-4fc5-8291-5ad86c073f70 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:26.668587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e196eb-816c-4c54-ada5-bbace363aa23 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:26.703876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fab86e-f59c-41ce-ac4c-7d58f9fac6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:26.704473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fab86e-f59c-41ce-ac4c-7d58f9fac6ee None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:26.704740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fab86e-f59c-41ce-ac4c-7d58f9fac6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:26.705031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fab86e-f59c-41ce-ac4c-7d58f9fac6ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:26.705747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fab86e-f59c-41ce-ac4c-7d58f9fac6ee None admin] RBAC: Adding query filter params (role.id=c54d8e4858914575b2ba91acf3233251, scope.project.id=931647de90cc4037a03cac17bd65f53d, user.id=fc7d6b6a272e4b9e8c23e879e7ab47da) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:26.706046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fab86e-f59c-41ce-ac4c-7d58f9fac6ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:26.707511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fab86e-f59c-41ce-ac4c-7d58f9fac6ee None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:26.713794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 126/254] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:26 2022] GET /identity/v3/role_assignments?user.id=fc7d6b6a272e4b9e8c23e879e7ab47da&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=c54d8e4858914575b2ba91acf3233251 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:26.714980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146521e-8c1e-40f0-86f1-a0bbbe8051e2 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:26.715256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146521e-8c1e-40f0-86f1-a0bbbe8051e2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:26.715525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146521e-8c1e-40f0-86f1-a0bbbe8051e2 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:26.716737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 127/255] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:26.722562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a241-aba9-4eae-87ec-65204a9f9349 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:26.722745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a241-aba9-4eae-87ec-65204a9f9349 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:26.729666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a241-aba9-4eae-87ec-65204a9f9349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:26.738871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 129/256] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:11:26 2022] PUT /identity/v3/projects/931647de90cc4037a03cac17bd65f53d/users/87d9337ae63047969a1c87e508676a08/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:26.742534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-4624a241-aba9-4eae-87ec-65204a9f9349 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:26.743762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4624a241-aba9-4eae-87ec-65204a9f9349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:26.766728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b81c1-354c-4fc5-8291-5ad86c073f70 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:26.777321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:26.865827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 128/257] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:26.917233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:27.076776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b81c1-354c-4fc5-8291-5ad86c073f70 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:27.146660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89415f90-7be0-4d76-9045-d73f429d4dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:27.147229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89415f90-7be0-4d76-9045-d73f429d4dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:27.147509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89415f90-7be0-4d76-9045-d73f429d4dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:27.147779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89415f90-7be0-4d76-9045-d73f429d4dc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:27.163088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89415f90-7be0-4d76-9045-d73f429d4dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-89415f90-7be0-4d76-9045-d73f429d4dc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:27.174898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:27.175824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:27.176588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:27.184156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 130/258] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:27.192820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a241-aba9-4eae-87ec-65204a9f9349 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:27.375661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a241-aba9-4eae-87ec-65204a9f9349 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:27.414299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89415f90-7be0-4d76-9045-d73f429d4dc6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:27.420908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24118-fde6-4f46-b631-63a04079a1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:27.421525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24118-fde6-4f46-b631-63a04079a1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:27.421802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24118-fde6-4f46-b631-63a04079a1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:27.422076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24118-fde6-4f46-b631-63a04079a1d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:27.422741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda24118-fde6-4f46-b631-63a04079a1d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:27.423040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda24118-fde6-4f46-b631-63a04079a1d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:27.424427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda24118-fde6-4f46-b631-63a04079a1d6 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:27.433621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 129/259] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:27.438914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24118-fde6-4f46-b631-63a04079a1d6 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:27.473329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:27.593313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24118-fde6-4f46-b631-63a04079a1d6 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:27.611206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89415f90-7be0-4d76-9045-d73f429d4dc6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:27.629821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:27.659884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc929a-5650-4527-8f68-83de82fb2314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:27.660967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc929a-5650-4527-8f68-83de82fb2314 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:27.661634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc929a-5650-4527-8f68-83de82fb2314 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:27.662316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc929a-5650-4527-8f68-83de82fb2314 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:27.674192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc929a-5650-4527-8f68-83de82fb2314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6fcc929a-5650-4527-8f68-83de82fb2314 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:27.679580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:27.680613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:27.681892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:27.683188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 131/260] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:27.683620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b8a92-4bd5-47ad-b18b-b38187598524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:27.684521 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b8a92-4bd5-47ad-b18b-b38187598524 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:27.684953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b8a92-4bd5-47ad-b18b-b38187598524 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:27.685343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b8a92-4bd5-47ad-b18b-b38187598524 None admin] PATH_INFO: `/v3/users/61705f82de4d4a0fb0b19ec86ea833e5` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:27.688179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b8a92-4bd5-47ad-b18b-b38187598524 None admin] RBAC: Authorizing `identity:get_user(user_id=61705f82de4d4a0fb0b19ec86ea833e5)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:27.688826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc929a-5650-4527-8f68-83de82fb2314 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:27.690003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b8a92-4bd5-47ad-b18b-b38187598524 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:27.696936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 130/261] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:27 2022] GET /identity/v3/users/61705f82de4d4a0fb0b19ec86ea833e5 => generated 283 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:27.705212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b8a92-4bd5-47ad-b18b-b38187598524 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:27.894982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:27.911612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc929a-5650-4527-8f68-83de82fb2314 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:27.913674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b8a92-4bd5-47ad-b18b-b38187598524 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:27.971457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0f6a6-69ec-410a-9947-6bd02c54acde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:27.971992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0f6a6-69ec-410a-9947-6bd02c54acde None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:27.972267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0f6a6-69ec-410a-9947-6bd02c54acde None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:27.972532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0f6a6-69ec-410a-9947-6bd02c54acde None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:27.973242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0f6a6-69ec-410a-9947-6bd02c54acde None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:27.973535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0f6a6-69ec-410a-9947-6bd02c54acde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:27.974903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0f6a6-69ec-410a-9947-6bd02c54acde None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:27.981853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07cd81-48df-47cc-8f81-437f8ae75dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:27.982719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07cd81-48df-47cc-8f81-437f8ae75dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:27.983178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07cd81-48df-47cc-8f81-437f8ae75dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:27.983622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07cd81-48df-47cc-8f81-437f8ae75dc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:27.983878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 132/262] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:27.989745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0f6a6-69ec-410a-9947-6bd02c54acde None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:28.063620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:28.108874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07cd81-48df-47cc-8f81-437f8ae75dc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:28.110347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07cd81-48df-47cc-8f81-437f8ae75dc8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:28.182991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0f6a6-69ec-410a-9947-6bd02c54acde None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:28.251429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd654da3-10fd-437c-9a7e-1f18470dd4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:28.252195 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd654da3-10fd-437c-9a7e-1f18470dd4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:28.252468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd654da3-10fd-437c-9a7e-1f18470dd4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:28.252856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd654da3-10fd-437c-9a7e-1f18470dd4a3 None admin] PATH_INFO: `/v3/users/61705f82de4d4a0fb0b19ec86ea833e5` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:28.257474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd654da3-10fd-437c-9a7e-1f18470dd4a3 None admin] RBAC: Authorizing `identity:get_user(user_id=61705f82de4d4a0fb0b19ec86ea833e5)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:28.258939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd654da3-10fd-437c-9a7e-1f18470dd4a3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:28.261376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 133/263] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:27 2022] GET /identity/v3/users/61705f82de4d4a0fb0b19ec86ea833e5 => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:28.262741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:28.268106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd654da3-10fd-437c-9a7e-1f18470dd4a3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:28.272418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 131/264] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:11:28.279390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07cd81-48df-47cc-8f81-437f8ae75dc8 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:28.440121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07cd81-48df-47cc-8f81-437f8ae75dc8 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:28.461471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd654da3-10fd-437c-9a7e-1f18470dd4a3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:28.482723 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:28.506966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44791d9b-7002-4ed4-8d65-03671cd7184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:28.507525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44791d9b-7002-4ed4-8d65-03671cd7184e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:28.507803 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44791d9b-7002-4ed4-8d65-03671cd7184e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:28.508074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44791d9b-7002-4ed4-8d65-03671cd7184e None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:28.513539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44791d9b-7002-4ed4-8d65-03671cd7184e None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:28.515020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44791d9b-7002-4ed4-8d65-03671cd7184e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:28.517186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 132/265] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:28 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:28.523230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44791d9b-7002-4ed4-8d65-03671cd7184e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:28.524139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4e776-b3cd-4b84-a168-cbdf710e39bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:28.524753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4e776-b3cd-4b84-a168-cbdf710e39bc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:28.525053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4e776-b3cd-4b84-a168-cbdf710e39bc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:28.525487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4e776-b3cd-4b84-a168-cbdf710e39bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:28.637428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:28.680259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4e776-b3cd-4b84-a168-cbdf710e39bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:28.683007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4e776-b3cd-4b84-a168-cbdf710e39bc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:28.689803 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44791d9b-7002-4ed4-8d65-03671cd7184e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:28.730490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5ab8b-8b7c-45a6-b260-9595a7024114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:28.731644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5ab8b-8b7c-45a6-b260-9595a7024114 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:28.732577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5ab8b-8b7c-45a6-b260-9595a7024114 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:28.733068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5ab8b-8b7c-45a6-b260-9595a7024114 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:28.733974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5ab8b-8b7c-45a6-b260-9595a7024114 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=931647de90cc4037a03cac17bd65f53d, user.id=61705f82de4d4a0fb0b19ec86ea833e5) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:28.734455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5ab8b-8b7c-45a6-b260-9595a7024114 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:28.736075 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5ab8b-8b7c-45a6-b260-9595a7024114 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:28.744098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 133/266] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:28 2022] GET /identity/v3/role_assignments?user.id=61705f82de4d4a0fb0b19ec86ea833e5&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:28.746275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbccc19-1f76-4b71-96aa-c70ab1109472 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:28.746742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbccc19-1f76-4b71-96aa-c70ab1109472 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:28.747292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbccc19-1f76-4b71-96aa-c70ab1109472 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:28.749300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 134/267] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:28.756228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30827cb6-5c9f-435f-8ba6-ec206a633f8c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:28.756812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30827cb6-5c9f-435f-8ba6-ec206a633f8c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:28.757323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30827cb6-5c9f-435f-8ba6-ec206a633f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:28.779420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-30827cb6-5c9f-435f-8ba6-ec206a633f8c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:28.780940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-30827cb6-5c9f-435f-8ba6-ec206a633f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:28.845583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 134/268] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:11:28.851540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4e776-b3cd-4b84-a168-cbdf710e39bc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:28.861179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:28.965736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4e776-b3cd-4b84-a168-cbdf710e39bc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:28.994625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 135/269] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:29.010177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e0130-1ce5-4d4a-a3fc-c6a48c4f5d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:29.010771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e0130-1ce5-4d4a-a3fc-c6a48c4f5d1a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.011055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e0130-1ce5-4d4a-a3fc-c6a48c4f5d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.011321 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e0130-1ce5-4d4a-a3fc-c6a48c4f5d1a None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.012549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e0130-1ce5-4d4a-a3fc-c6a48c4f5d1a None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:29.014022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e0130-1ce5-4d4a-a3fc-c6a48c4f5d1a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:29.015829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 135/270] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:28 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:29.021570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30827cb6-5c9f-435f-8ba6-ec206a633f8c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:29.113761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:29.146676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14749a4b-2353-4cb9-8dd3-f0197d4e4837 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.147007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14749a4b-2353-4cb9-8dd3-f0197d4e4837 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.147226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14749a4b-2353-4cb9-8dd3-f0197d4e4837 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.149259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 136/271] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:29.156965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7abd1-6455-471a-a243-46ddf78600c8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.157235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7abd1-6455-471a-a243-46ddf78600c8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.157509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7abd1-6455-471a-a243-46ddf78600c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.175212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30827cb6-5c9f-435f-8ba6-ec206a633f8c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:29.179559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-43f7abd1-6455-471a-a243-46ddf78600c8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:29.180746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-43f7abd1-6455-471a-a243-46ddf78600c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:29.256853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d6fb0-dba5-47fa-b347-47bbd5cd4b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:29.257483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d6fb0-dba5-47fa-b347-47bbd5cd4b01 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.257761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d6fb0-dba5-47fa-b347-47bbd5cd4b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.258030 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d6fb0-dba5-47fa-b347-47bbd5cd4b01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.258756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d6fb0-dba5-47fa-b347-47bbd5cd4b01 None admin] RBAC: Adding query filter params (role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=931647de90cc4037a03cac17bd65f53d, user.id=61705f82de4d4a0fb0b19ec86ea833e5) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:29.259052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d6fb0-dba5-47fa-b347-47bbd5cd4b01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:29.260777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d6fb0-dba5-47fa-b347-47bbd5cd4b01 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:29.262102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 137/272] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:29.267718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 136/273] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:29 2022] GET /identity/v3/role_assignments?user.id=61705f82de4d4a0fb0b19ec86ea833e5&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:29.304111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7abd1-6455-471a-a243-46ddf78600c8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:29.308533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:29.442535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7abd1-6455-471a-a243-46ddf78600c8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:29.501636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3263f952-4148-4f69-9639-cf58559efd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:29.503022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263f952-4148-4f69-9639-cf58559efd41 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.503448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263f952-4148-4f69-9639-cf58559efd41 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.503865 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263f952-4148-4f69-9639-cf58559efd41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.517074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3263f952-4148-4f69-9639-cf58559efd41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3263f952-4148-4f69-9639-cf58559efd41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:29.522139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:29.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:29.523763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 138/274] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:29.529737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d6fb0-dba5-47fa-b347-47bbd5cd4b01 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:29.641252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d6fb0-dba5-47fa-b347-47bbd5cd4b01 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:29.663150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:29.681029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc83af3-84c9-4258-8750-2f624ad30702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:29.682335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc83af3-84c9-4258-8750-2f624ad30702 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.682702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc83af3-84c9-4258-8750-2f624ad30702 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.682968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc83af3-84c9-4258-8750-2f624ad30702 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.683662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc83af3-84c9-4258-8750-2f624ad30702 None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:29.683962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc83af3-84c9-4258-8750-2f624ad30702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:29.685363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc83af3-84c9-4258-8750-2f624ad30702 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:29.693344 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927567c-cfb6-4a11-b712-c7ae5c1ea4a3 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.693690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927567c-cfb6-4a11-b712-c7ae5c1ea4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.693967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927567c-cfb6-4a11-b712-c7ae5c1ea4a3 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.695282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 139/275] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:29.695977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927567c-cfb6-4a11-b712-c7ae5c1ea4a3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:29.698151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 137/276] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:29.702878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc83af3-84c9-4258-8750-2f624ad30702 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:29.777862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:29.809375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc83af3-84c9-4258-8750-2f624ad30702 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:29.820720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927567c-cfb6-4a11-b712-c7ae5c1ea4a3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:29.843837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7fd56-0bff-4baf-be1a-dea9b1b94c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:29.844473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7fd56-0bff-4baf-be1a-dea9b1b94c6c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.844744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7fd56-0bff-4baf-be1a-dea9b1b94c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.845050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7fd56-0bff-4baf-be1a-dea9b1b94c6c None admin] PATH_INFO: `/v3/users/d51637a5abf94eaea17f462b29cbbadf` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.856806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5d4ef-7b51-4f18-ac53-ba4e03927fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:29.857445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5d4ef-7b51-4f18-ac53-ba4e03927fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.857722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5d4ef-7b51-4f18-ac53-ba4e03927fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.857991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5d4ef-7b51-4f18-ac53-ba4e03927fc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.859088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5d4ef-7b51-4f18-ac53-ba4e03927fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:29.860545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5d4ef-7b51-4f18-ac53-ba4e03927fc0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:29.860980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7fd56-0bff-4baf-be1a-dea9b1b94c6c None admin] RBAC: Authorizing `identity:get_user(user_id=d51637a5abf94eaea17f462b29cbbadf)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:29.862138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 140/277] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:29.862787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7fd56-0bff-4baf-be1a-dea9b1b94c6c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:29.863353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333097-56d6-4a51-9ff7-c23892f25da4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:29.863631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333097-56d6-4a51-9ff7-c23892f25da4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:29.863901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333097-56d6-4a51-9ff7-c23892f25da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:29.865175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 138/278] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:29 2022] GET /identity/v3/users/d51637a5abf94eaea17f462b29cbbadf => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:29.867555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7fd56-0bff-4baf-be1a-dea9b1b94c6c None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:29.884598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8b333097-56d6-4a51-9ff7-c23892f25da4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:29.885805 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8b333097-56d6-4a51-9ff7-c23892f25da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:29.914784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:29.964742 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 141/279] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:29.966033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333097-56d6-4a51-9ff7-c23892f25da4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:29.988200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7fd56-0bff-4baf-be1a-dea9b1b94c6c None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:30.025382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4a739-5ee5-4225-86f7-2ecd304d74ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:30.026128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4a739-5ee5-4225-86f7-2ecd304d74ff None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.026406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4a739-5ee5-4225-86f7-2ecd304d74ff None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.026673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4a739-5ee5-4225-86f7-2ecd304d74ff None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.027416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4a739-5ee5-4225-86f7-2ecd304d74ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:30.029043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4a739-5ee5-4225-86f7-2ecd304d74ff None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:30.029893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:30.068678 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333097-56d6-4a51-9ff7-c23892f25da4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:30.102051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df13021-56bb-452f-a9a1-507b1498da1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:30.102644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df13021-56bb-452f-a9a1-507b1498da1a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.102946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df13021-56bb-452f-a9a1-507b1498da1a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.103916 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df13021-56bb-452f-a9a1-507b1498da1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.104624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df13021-56bb-452f-a9a1-507b1498da1a None admin] RBAC: Adding query filter params (role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, user.id=d51637a5abf94eaea17f462b29cbbadf, scope.system=all) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:30.104961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df13021-56bb-452f-a9a1-507b1498da1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:30.106647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df13021-56bb-452f-a9a1-507b1498da1a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:30.111504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 142/280] 10.222.0.84 () {56 vars in 1308 bytes} [Wed Aug 24 11:11:29 2022] GET /identity/v3/role_assignments?user.id=d51637a5abf94eaea17f462b29cbbadf&scope.system=all&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:30.112889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149597f-ef6e-4c7b-aaf3-de0ebd172957 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.113166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149597f-ef6e-4c7b-aaf3-de0ebd172957 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.113434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149597f-ef6e-4c7b-aaf3-de0ebd172957 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.114837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 143/281] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:30.121542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003bba6-55e5-4989-a5ed-a54333feaa54 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.121878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003bba6-55e5-4989-a5ed-a54333feaa54 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.126078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003bba6-55e5-4989-a5ed-a54333feaa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.142051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8003bba6-55e5-4989-a5ed-a54333feaa54 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:30.143258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8003bba6-55e5-4989-a5ed-a54333feaa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:30.175675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 139/282] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:29 2022] POST /identity/v3/users => generated 293 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:11:30.235870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 144/283] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:30.246194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4a739-5ee5-4225-86f7-2ecd304d74ff None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:30.356619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:30.365120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4a739-5ee5-4225-86f7-2ecd304d74ff None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:30.398860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd98f5-55f5-4622-a25f-47121a2d3a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:30.399555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd98f5-55f5-4622-a25f-47121a2d3a2d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.399834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd98f5-55f5-4622-a25f-47121a2d3a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.400695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd98f5-55f5-4622-a25f-47121a2d3a2d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.412449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd98f5-55f5-4622-a25f-47121a2d3a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-25fd98f5-55f5-4622-a25f-47121a2d3a2d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:30.418115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:30.422830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:30.423630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 140/284] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:30.429908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003bba6-55e5-4989-a5ed-a54333feaa54 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:30.490035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:30.514309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd98f5-55f5-4622-a25f-47121a2d3a2d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:30.542320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003bba6-55e5-4989-a5ed-a54333feaa54 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:30.587160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787e39-fdb2-4a64-9460-60382580200f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:30.588597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787e39-fdb2-4a64-9460-60382580200f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.588895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787e39-fdb2-4a64-9460-60382580200f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.589161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787e39-fdb2-4a64-9460-60382580200f None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.589820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787e39-fdb2-4a64-9460-60382580200f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:30.590171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787e39-fdb2-4a64-9460-60382580200f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:30.591787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787e39-fdb2-4a64-9460-60382580200f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:30.602681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 145/285] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:30.608903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787e39-fdb2-4a64-9460-60382580200f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:30.646772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd98f5-55f5-4622-a25f-47121a2d3a2d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:30.694283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1174-741e-43a5-8c69-81b054a20e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:30.694283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1174-741e-43a5-8c69-81b054a20e93 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.694549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1174-741e-43a5-8c69-81b054a20e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.694822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c1174-741e-43a5-8c69-81b054a20e93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.706617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c1174-741e-43a5-8c69-81b054a20e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-220c1174-741e-43a5-8c69-81b054a20e93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:30.712235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:30.713178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:30.713934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 141/286] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:11:30.721988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1174-741e-43a5-8c69-81b054a20e93 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:30.732375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787e39-fdb2-4a64-9460-60382580200f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:30.748691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:30.751910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:30.770725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c71df2-d72f-4385-984b-9e96388312e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:30.770998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c71df2-d72f-4385-984b-9e96388312e2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.771373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c71df2-d72f-4385-984b-9e96388312e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.771741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c71df2-d72f-4385-984b-9e96388312e2 None admin] PATH_INFO: `/v3/users/fc7d6b6a272e4b9e8c23e879e7ab47da` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.773032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c71df2-d72f-4385-984b-9e96388312e2 None admin] RBAC: Authorizing `identity:get_user(user_id=fc7d6b6a272e4b9e8c23e879e7ab47da)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:30.774579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c71df2-d72f-4385-984b-9e96388312e2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:30.776958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 146/287] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:30 2022] GET /identity/v3/users/fc7d6b6a272e4b9e8c23e879e7ab47da => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:30.782768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c71df2-d72f-4385-984b-9e96388312e2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:30.851905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c1174-741e-43a5-8c69-81b054a20e93 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:30.892574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c71df2-d72f-4385-984b-9e96388312e2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:30.895861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805326c-8f37-44d2-abdf-34385125cba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:30.896659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805326c-8f37-44d2-abdf-34385125cba9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.897110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805326c-8f37-44d2-abdf-34385125cba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.897524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805326c-8f37-44d2-abdf-34385125cba9 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.898323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805326c-8f37-44d2-abdf-34385125cba9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:30.898762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805326c-8f37-44d2-abdf-34385125cba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:30.901260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805326c-8f37-44d2-abdf-34385125cba9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:30.914830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 142/288] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:30.921752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805326c-8f37-44d2-abdf-34385125cba9 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:30.932271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be118c81-3426-491a-9b39-084cc9f62c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:30.940821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be118c81-3426-491a-9b39-084cc9f62c5d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:30.940821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be118c81-3426-491a-9b39-084cc9f62c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:30.940821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be118c81-3426-491a-9b39-084cc9f62c5d None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:30.940821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be118c81-3426-491a-9b39-084cc9f62c5d None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:30.940821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be118c81-3426-491a-9b39-084cc9f62c5d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:30.940821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 147/289] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:30 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:30.943953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be118c81-3426-491a-9b39-084cc9f62c5d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:31.009152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:31.055933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be118c81-3426-491a-9b39-084cc9f62c5d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:31.090491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:31.095223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff417a-6a95-4bea-900d-715e7be3eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:31.095892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff417a-6a95-4bea-900d-715e7be3eaa2 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:31.096168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff417a-6a95-4bea-900d-715e7be3eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:31.096439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff417a-6a95-4bea-900d-715e7be3eaa2 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d/users/fc7d6b6a272e4b9e8c23e879e7ab47da/roles/c54d8e4858914575b2ba91acf3233251` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:31.101430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff417a-6a95-4bea-900d-715e7be3eaa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=931647de90cc4037a03cac17bd65f53d, user_id=fc7d6b6a272e4b9e8c23e879e7ab47da, role_id=c54d8e4858914575b2ba91acf3233251)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:31.104481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff417a-6a95-4bea-900d-715e7be3eaa2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:31.124514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805326c-8f37-44d2-abdf-34385125cba9 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:31.164000 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0060ca-45ff-4cc6-9252-349abe000dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:31.164556 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0060ca-45ff-4cc6-9252-349abe000dfb None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:31.164859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0060ca-45ff-4cc6-9252-349abe000dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:31.165132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0060ca-45ff-4cc6-9252-349abe000dfb None admin] PATH_INFO: `/v3/users/87d9337ae63047969a1c87e508676a08` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:31.166373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0060ca-45ff-4cc6-9252-349abe000dfb None admin] RBAC: Authorizing `identity:get_user(user_id=87d9337ae63047969a1c87e508676a08)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:31.167928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0060ca-45ff-4cc6-9252-349abe000dfb None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:31.169938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 143/290] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:30 2022] GET /identity/v3/users/87d9337ae63047969a1c87e508676a08 => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:31.177383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0060ca-45ff-4cc6-9252-349abe000dfb None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:31.198501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 148/291] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:11:30 2022] PUT /identity/v3/projects/931647de90cc4037a03cac17bd65f53d/users/fc7d6b6a272e4b9e8c23e879e7ab47da/roles/c54d8e4858914575b2ba91acf3233251 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:31.314813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0060ca-45ff-4cc6-9252-349abe000dfb None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:31.360602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc2f8a-5118-41fd-803e-b9d1759a0700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:31.361441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc2f8a-5118-41fd-803e-b9d1759a0700 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:31.361623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc2f8a-5118-41fd-803e-b9d1759a0700 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:31.361867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc2f8a-5118-41fd-803e-b9d1759a0700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:31.388869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:31.456541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc2f8a-5118-41fd-803e-b9d1759a0700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:31.458022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc2f8a-5118-41fd-803e-b9d1759a0700 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:31.565716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 144/292] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:31.571288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff417a-6a95-4bea-900d-715e7be3eaa2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:31.711575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff417a-6a95-4bea-900d-715e7be3eaa2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:31.734166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:31.757245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e2e2b-f3d1-4007-9c18-a575d17bc0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:31.757884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e2e2b-f3d1-4007-9c18-a575d17bc0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:31.758156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e2e2b-f3d1-4007-9c18-a575d17bc0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:31.758427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e2e2b-f3d1-4007-9c18-a575d17bc0e1 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:31.765575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e2e2b-f3d1-4007-9c18-a575d17bc0e1 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:31.767048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e2e2b-f3d1-4007-9c18-a575d17bc0e1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:31.769099 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 149/293] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:31 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:31.774795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc2f8a-5118-41fd-803e-b9d1759a0700 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:31.829195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:31.881882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc2f8a-5118-41fd-803e-b9d1759a0700 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:31.918305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7662d2f7-ccc6-4c50-9e6e-dfa37828b754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:31.919613 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662d2f7-ccc6-4c50-9e6e-dfa37828b754 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:31.919732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662d2f7-ccc6-4c50-9e6e-dfa37828b754 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:31.921486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662d2f7-ccc6-4c50-9e6e-dfa37828b754 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:31.921486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662d2f7-ccc6-4c50-9e6e-dfa37828b754 None admin] RBAC: Adding query filter params (role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=931647de90cc4037a03cac17bd65f53d, user.id=87d9337ae63047969a1c87e508676a08) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:31.921486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662d2f7-ccc6-4c50-9e6e-dfa37828b754 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:31.924022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7662d2f7-ccc6-4c50-9e6e-dfa37828b754 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:31.935179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 145/294] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:31 2022] GET /identity/v3/role_assignments?user.id=87d9337ae63047969a1c87e508676a08&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:32.340072 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc4398-e602-4420-b265-24048c9eec1c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:32.340474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc4398-e602-4420-b265-24048c9eec1c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:32.340651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc4398-e602-4420-b265-24048c9eec1c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:32.342238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 150/295] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:32.350325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed4ff6-2bab-4922-8f36-013f9a4e6677 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:32.350612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed4ff6-2bab-4922-8f36-013f9a4e6677 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:32.350883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed4ff6-2bab-4922-8f36-013f9a4e6677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:32.371134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d3ed4ff6-2bab-4922-8f36-013f9a4e6677 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:32.372342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d3ed4ff6-2bab-4922-8f36-013f9a4e6677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:32.445269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 146/296] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:32.726992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc4398-e602-4420-b265-24048c9eec1c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:32.838208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc4398-e602-4420-b265-24048c9eec1c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:32.843979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ea236-1325-4ea4-add6-fe440c7957df None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:32.844289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ea236-1325-4ea4-add6-fe440c7957df None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:32.844572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ea236-1325-4ea4-add6-fe440c7957df None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:32.846191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 147/297] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:32.851748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd9dd-d0b1-4d01-864b-9704c790c1a5 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:32.852258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd9dd-d0b1-4d01-864b-9704c790c1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:32.852992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd9dd-d0b1-4d01-864b-9704c790c1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:32.874106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c9bcd9dd-d0b1-4d01-864b-9704c790c1a5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:32.875511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c9bcd9dd-d0b1-4d01-864b-9704c790c1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:32.875663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7c9c5-9c38-401a-b7e0-d87a8ea838e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:32.876136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c9c5-9c38-401a-b7e0-d87a8ea838e1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:32.876408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c9c5-9c38-401a-b7e0-d87a8ea838e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:32.876673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c9c5-9c38-401a-b7e0-d87a8ea838e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:32.887494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7c9c5-9c38-401a-b7e0-d87a8ea838e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d2b7c9c5-9c38-401a-b7e0-d87a8ea838e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:32.892562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:32.893458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:32.894222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 151/298] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:32.894857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0456a83-1765-438d-852f-0c3105a99703 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:32.895366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0456a83-1765-438d-852f-0c3105a99703 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:32.895366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0456a83-1765-438d-852f-0c3105a99703 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:32.896694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 152/299] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:32.901534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0456a83-1765-438d-852f-0c3105a99703 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:32.917602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:32.949713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:32.951879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 148/300] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:32.959047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d386f8-9cce-40d9-82df-26d64db5fd87 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:32.959996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d386f8-9cce-40d9-82df-26d64db5fd87 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:32.960452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d386f8-9cce-40d9-82df-26d64db5fd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:32.990030 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-51d386f8-9cce-40d9-82df-26d64db5fd87 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:32.992108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-51d386f8-9cce-40d9-82df-26d64db5fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:33.016179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0456a83-1765-438d-852f-0c3105a99703 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:33.052827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a380c-20df-41cd-92ec-edb7707c3d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:33.053709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a380c-20df-41cd-92ec-edb7707c3d21 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:33.054048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a380c-20df-41cd-92ec-edb7707c3d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:33.054428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a380c-20df-41cd-92ec-edb7707c3d21 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:33.055155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a380c-20df-41cd-92ec-edb7707c3d21 None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:33.055743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a380c-20df-41cd-92ec-edb7707c3d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:33.057655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a380c-20df-41cd-92ec-edb7707c3d21 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:33.066890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 153/301] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:33.072067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a380c-20df-41cd-92ec-edb7707c3d21 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:33.091627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 149/302] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:33.184317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a380c-20df-41cd-92ec-edb7707c3d21 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:33.186961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:33.219550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8774d-77d9-4a0c-8d63-fbe35be51a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:33.220181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8774d-77d9-4a0c-8d63-fbe35be51a90 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:33.220453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8774d-77d9-4a0c-8d63-fbe35be51a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:33.220725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8774d-77d9-4a0c-8d63-fbe35be51a90 None admin] PATH_INFO: `/v3/users/d51637a5abf94eaea17f462b29cbbadf` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:33.221945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8774d-77d9-4a0c-8d63-fbe35be51a90 None admin] RBAC: Authorizing `identity:get_user(user_id=d51637a5abf94eaea17f462b29cbbadf)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:33.223393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8774d-77d9-4a0c-8d63-fbe35be51a90 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:33.225160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 154/303] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:33 2022] GET /identity/v3/users/d51637a5abf94eaea17f462b29cbbadf => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:33.231882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d386f8-9cce-40d9-82df-26d64db5fd87 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:33.235229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8774d-77d9-4a0c-8d63-fbe35be51a90 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:33.342483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8774d-77d9-4a0c-8d63-fbe35be51a90 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:33.354894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:33.359004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d386f8-9cce-40d9-82df-26d64db5fd87 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:33.379544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d0bb9-aff2-48af-bc92-d637f015516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:33.380541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d0bb9-aff2-48af-bc92-d637f015516d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:33.381096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d0bb9-aff2-48af-bc92-d637f015516d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:33.381566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d0bb9-aff2-48af-bc92-d637f015516d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:33.382863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d0bb9-aff2-48af-bc92-d637f015516d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:33.384654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d0bb9-aff2-48af-bc92-d637f015516d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:33.386839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 155/304] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:33.393296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d0bb9-aff2-48af-bc92-d637f015516d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:33.401181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f438ed-be7c-456e-8b65-0d81c17a5d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:33.401938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f438ed-be7c-456e-8b65-0d81c17a5d72 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:33.402251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f438ed-be7c-456e-8b65-0d81c17a5d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:33.402569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f438ed-be7c-456e-8b65-0d81c17a5d72 None admin] PATH_INFO: `/v3/system/users/d51637a5abf94eaea17f462b29cbbadf/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:33.405004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f438ed-be7c-456e-8b65-0d81c17a5d72 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d51637a5abf94eaea17f462b29cbbadf, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:33.407384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f438ed-be7c-456e-8b65-0d81c17a5d72 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:33.436051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:33.493209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d0bb9-aff2-48af-bc92-d637f015516d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:33.495553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 150/305] 10.222.0.84 () {58 vars in 1334 bytes} [Wed Aug 24 11:11:33 2022] PUT /identity/v3/system/users/d51637a5abf94eaea17f462b29cbbadf/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:33.496500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f438ed-be7c-456e-8b65-0d81c17a5d72 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:33.542951 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f52abb-bcfe-4bbf-9092-560b56f85c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:33.544126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f52abb-bcfe-4bbf-9092-560b56f85c87 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:33.544444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f52abb-bcfe-4bbf-9092-560b56f85c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:33.544759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f52abb-bcfe-4bbf-9092-560b56f85c87 None admin] PATH_INFO: `/v3/groups` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:33.545559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f52abb-bcfe-4bbf-9092-560b56f85c87 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:33.547281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f52abb-bcfe-4bbf-9092-560b56f85c87 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:33.616089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f438ed-be7c-456e-8b65-0d81c17a5d72 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:33.660581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077db247-79e2-440e-b3f0-3233e5cefadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:33.661261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077db247-79e2-440e-b3f0-3233e5cefadf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:33.661550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077db247-79e2-440e-b3f0-3233e5cefadf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:33.661815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077db247-79e2-440e-b3f0-3233e5cefadf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:33.672987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077db247-79e2-440e-b3f0-3233e5cefadf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-077db247-79e2-440e-b3f0-3233e5cefadf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:33.678175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:33.679235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:33.680215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 151/306] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:33.681072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e5393-2882-4d99-a0d5-85895d55e8fd None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:33.681072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e5393-2882-4d99-a0d5-85895d55e8fd None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:33.681072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e5393-2882-4d99-a0d5-85895d55e8fd None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:33.682948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 152/307] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:33.687922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e5393-2882-4d99-a0d5-85895d55e8fd None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:33.688717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 156/308] 10.222.0.84 () {60 vars in 1134 bytes} [Wed Aug 24 11:11:33 2022] POST /identity/v3/groups => generated 229 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:33.690143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c03f0-738b-4e03-9fa8-dea4c3397390 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:33.707333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c03f0-738b-4e03-9fa8-dea4c3397390 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:33.707333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c03f0-738b-4e03-9fa8-dea4c3397390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:33.709885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-621c03f0-738b-4e03-9fa8-dea4c3397390 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:33.711099 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-621c03f0-738b-4e03-9fa8-dea4c3397390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:33.715972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:33.808099 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 157/309] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:33.810715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e5393-2882-4d99-a0d5-85895d55e8fd None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:33.848696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306029b9-d4f0-464d-93ef-18da127f2657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:33.849360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306029b9-d4f0-464d-93ef-18da127f2657 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:33.849644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306029b9-d4f0-464d-93ef-18da127f2657 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:33.849919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306029b9-d4f0-464d-93ef-18da127f2657 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:33.850599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306029b9-d4f0-464d-93ef-18da127f2657 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:33.850904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306029b9-d4f0-464d-93ef-18da127f2657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:33.852423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306029b9-d4f0-464d-93ef-18da127f2657 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:33.864696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 153/310] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:33.870482 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c03f0-738b-4e03-9fa8-dea4c3397390 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:33.902765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:33.985443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c03f0-738b-4e03-9fa8-dea4c3397390 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:34.021372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a4cb1-8afc-44b1-aea0-4bb5e38e9c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:34.022003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a4cb1-8afc-44b1-aea0-4bb5e38e9c2e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:34.022332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a4cb1-8afc-44b1-aea0-4bb5e38e9c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:34.022615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a4cb1-8afc-44b1-aea0-4bb5e38e9c2e None admin] PATH_INFO: `/v3/users/aefef4c62280482582e7804465a9a0eb` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:34.036907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a4cb1-8afc-44b1-aea0-4bb5e38e9c2e None admin] RBAC: Authorizing `identity:get_user(user_id=aefef4c62280482582e7804465a9a0eb)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:34.038370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a4cb1-8afc-44b1-aea0-4bb5e38e9c2e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:34.040328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 158/311] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:33 2022] GET /identity/v3/users/aefef4c62280482582e7804465a9a0eb => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:34.046766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306029b9-d4f0-464d-93ef-18da127f2657 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:34.093247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a4cb1-8afc-44b1-aea0-4bb5e38e9c2e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:34.141325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:34.156278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306029b9-d4f0-464d-93ef-18da127f2657 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:34.194342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38cf4d-f20b-42e5-a8c6-e46e0d58d7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:34.195040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38cf4d-f20b-42e5-a8c6-e46e0d58d7fd None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:34.195384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38cf4d-f20b-42e5-a8c6-e46e0d58d7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:34.195722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38cf4d-f20b-42e5-a8c6-e46e0d58d7fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:34.196531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38cf4d-f20b-42e5-a8c6-e46e0d58d7fd None admin] RBAC: Adding query filter params (role.id=c54d8e4858914575b2ba91acf3233251, user.id=aefef4c62280482582e7804465a9a0eb, scope.system=all) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:34.197545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38cf4d-f20b-42e5-a8c6-e46e0d58d7fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:34.199061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38cf4d-f20b-42e5-a8c6-e46e0d58d7fd None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:34.211465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 154/312] 10.222.0.84 () {56 vars in 1308 bytes} [Wed Aug 24 11:11:34 2022] GET /identity/v3/role_assignments?user.id=aefef4c62280482582e7804465a9a0eb&scope.system=all&role.id=c54d8e4858914575b2ba91acf3233251 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:34.221592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:34.233158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a4cb1-8afc-44b1-aea0-4bb5e38e9c2e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:34.273896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13b9dd-6c87-4915-96bc-446fd85aef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:34.274457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13b9dd-6c87-4915-96bc-446fd85aef2c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:34.274731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13b9dd-6c87-4915-96bc-446fd85aef2c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:34.275035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13b9dd-6c87-4915-96bc-446fd85aef2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:34.287217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13b9dd-6c87-4915-96bc-446fd85aef2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-bc13b9dd-6c87-4915-96bc-446fd85aef2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:34.293026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:34.294008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:34.294787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 159/313] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:34.301960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38cf4d-f20b-42e5-a8c6-e46e0d58d7fd None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:34.413328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38cf4d-f20b-42e5-a8c6-e46e0d58d7fd None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:34.450704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a39033-a080-48ef-80db-cb6db602f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:34.451655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a39033-a080-48ef-80db-cb6db602f88f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:34.452157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a39033-a080-48ef-80db-cb6db602f88f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:34.452638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a39033-a080-48ef-80db-cb6db602f88f None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:34.453530 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a39033-a080-48ef-80db-cb6db602f88f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:34.454023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a39033-a080-48ef-80db-cb6db602f88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:34.455633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a39033-a080-48ef-80db-cb6db602f88f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:34.465754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 155/314] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:34.471480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13b9dd-6c87-4915-96bc-446fd85aef2c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:34.580974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13b9dd-6c87-4915-96bc-446fd85aef2c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:34.617129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4246c-cd10-4e7a-b083-caaf12c7b0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:34.618054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4246c-cd10-4e7a-b083-caaf12c7b0ea None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:34.618535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4246c-cd10-4e7a-b083-caaf12c7b0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:34.619521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4246c-cd10-4e7a-b083-caaf12c7b0ea None admin] PATH_INFO: `/v3/users/fc7d6b6a272e4b9e8c23e879e7ab47da` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:34.620859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4246c-cd10-4e7a-b083-caaf12c7b0ea None admin] RBAC: Authorizing `identity:get_user(user_id=fc7d6b6a272e4b9e8c23e879e7ab47da)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:34.622476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4246c-cd10-4e7a-b083-caaf12c7b0ea None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:34.624751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 160/315] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:34 2022] GET /identity/v3/users/fc7d6b6a272e4b9e8c23e879e7ab47da => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:34.630894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a39033-a080-48ef-80db-cb6db602f88f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:34.647297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:34.739790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a39033-a080-48ef-80db-cb6db602f88f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:34.774356 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7eea4-c8e7-47a7-bd12-585da4b69514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:34.775477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7eea4-c8e7-47a7-bd12-585da4b69514 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:34.775756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7eea4-c8e7-47a7-bd12-585da4b69514 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:34.776085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7eea4-c8e7-47a7-bd12-585da4b69514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:34.846599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7eea4-c8e7-47a7-bd12-585da4b69514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:34.848008 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7eea4-c8e7-47a7-bd12-585da4b69514 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:34.866229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:34.956916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 156/316] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:34.958595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4246c-cd10-4e7a-b083-caaf12c7b0ea None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:34.978727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:35.066700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4246c-cd10-4e7a-b083-caaf12c7b0ea None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:35.102264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e162f-b9d2-4b91-b20d-a82c2b04be92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:35.102823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e162f-b9d2-4b91-b20d-a82c2b04be92 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:35.103566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e162f-b9d2-4b91-b20d-a82c2b04be92 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:35.103844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e162f-b9d2-4b91-b20d-a82c2b04be92 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:35.105008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e162f-b9d2-4b91-b20d-a82c2b04be92 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:35.106805 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e162f-b9d2-4b91-b20d-a82c2b04be92 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:35.108511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 161/317] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:34 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:35.114641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7eea4-c8e7-47a7-bd12-585da4b69514 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:35.219935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7eea4-c8e7-47a7-bd12-585da4b69514 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:35.234457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:35.256272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa30a7-3e9d-42fc-8d18-67f2bd75d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:35.256904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa30a7-3e9d-42fc-8d18-67f2bd75d91a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:35.257213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa30a7-3e9d-42fc-8d18-67f2bd75d91a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:35.257444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa30a7-3e9d-42fc-8d18-67f2bd75d91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:35.258166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa30a7-3e9d-42fc-8d18-67f2bd75d91a None admin] RBAC: Adding query filter params (role.id=c54d8e4858914575b2ba91acf3233251, scope.project.id=931647de90cc4037a03cac17bd65f53d, user.id=fc7d6b6a272e4b9e8c23e879e7ab47da) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:35.258470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa30a7-3e9d-42fc-8d18-67f2bd75d91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:35.260445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa30a7-3e9d-42fc-8d18-67f2bd75d91a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:35.267044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 157/318] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:11:35 2022] GET /identity/v3/role_assignments?user.id=fc7d6b6a272e4b9e8c23e879e7ab47da&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=c54d8e4858914575b2ba91acf3233251 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:35.577681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710557-8aea-4f6d-a70d-61e4118e2978 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:35.578010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710557-8aea-4f6d-a70d-61e4118e2978 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:35.578241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3710557-8aea-4f6d-a70d-61e4118e2978 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:35.579616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 162/319] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:35.585057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79403e4c-f563-45db-b1db-2581a5559ea1 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:35.585495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79403e4c-f563-45db-b1db-2581a5559ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:35.585659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79403e4c-f563-45db-b1db-2581a5559ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:35.605437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-79403e4c-f563-45db-b1db-2581a5559ea1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:35.606864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-79403e4c-f563-45db-b1db-2581a5559ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:35.678925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 158/320] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:35.813446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29463b1-33aa-48bd-9756-6c4dec862bab None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:35.813837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29463b1-33aa-48bd-9756-6c4dec862bab None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:35.814012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29463b1-33aa-48bd-9756-6c4dec862bab None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:35.815464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 163/321] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:35.820448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b749329-264f-4f2a-aa55-ef972df69f67 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:35.820738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b749329-264f-4f2a-aa55-ef972df69f67 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:35.821047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b749329-264f-4f2a-aa55-ef972df69f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:35.840249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-7b749329-264f-4f2a-aa55-ef972df69f67 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:35.841370 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7b749329-264f-4f2a-aa55-ef972df69f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:35.909184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 159/322] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:35.975566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29463b1-33aa-48bd-9756-6c4dec862bab None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:36.064966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:36.082129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29463b1-33aa-48bd-9756-6c4dec862bab None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:36.120137 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706db5f-83d2-4c46-9559-ea644aee86f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:36.120684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706db5f-83d2-4c46-9559-ea644aee86f9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.121003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706db5f-83d2-4c46-9559-ea644aee86f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.121280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706db5f-83d2-4c46-9559-ea644aee86f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.131782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4706db5f-83d2-4c46-9559-ea644aee86f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-4706db5f-83d2-4c46-9559-ea644aee86f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:36.137310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:36.138177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:36.139174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 164/323] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:36.144051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9e96d-59e1-4c81-80b7-deb51a759ecb None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.144203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706db5f-83d2-4c46-9559-ea644aee86f9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:36.144380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9e96d-59e1-4c81-80b7-deb51a759ecb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.144594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9e96d-59e1-4c81-80b7-deb51a759ecb None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.146150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 160/324] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:36.151760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7f98e-07b2-475b-ae38-44f087389567 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.152044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7f98e-07b2-475b-ae38-44f087389567 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.152318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7f98e-07b2-475b-ae38-44f087389567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.170229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-5de7f98e-07b2-475b-ae38-44f087389567 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:36.171672 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5de7f98e-07b2-475b-ae38-44f087389567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:36.202133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:36.240747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 161/325] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:36.249055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706db5f-83d2-4c46-9559-ea644aee86f9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:36.249748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7f98e-07b2-475b-ae38-44f087389567 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:36.284024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38354fa-2dbb-4418-8dbf-e3679076452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:36.284620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38354fa-2dbb-4418-8dbf-e3679076452c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.284918 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38354fa-2dbb-4418-8dbf-e3679076452c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.285186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38354fa-2dbb-4418-8dbf-e3679076452c None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.285838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38354fa-2dbb-4418-8dbf-e3679076452c None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:36.286136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38354fa-2dbb-4418-8dbf-e3679076452c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:36.287494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38354fa-2dbb-4418-8dbf-e3679076452c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:36.295941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 165/326] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:36.301022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38354fa-2dbb-4418-8dbf-e3679076452c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:36.310750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:36.358572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7f98e-07b2-475b-ae38-44f087389567 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:36.392941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e1574-fbea-4759-b3ef-24bce2374106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:36.393494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e1574-fbea-4759-b3ef-24bce2374106 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.393765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e1574-fbea-4759-b3ef-24bce2374106 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.394030 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e1574-fbea-4759-b3ef-24bce2374106 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.395124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e1574-fbea-4759-b3ef-24bce2374106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:36.396603 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e1574-fbea-4759-b3ef-24bce2374106 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:36.400857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 162/327] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:36.404609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e1574-fbea-4759-b3ef-24bce2374106 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:36.421308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38354fa-2dbb-4418-8dbf-e3679076452c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:36.443333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:36.458692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f7c94-f656-427d-abad-053b4ec83c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:36.459320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f7c94-f656-427d-abad-053b4ec83c71 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.459592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f7c94-f656-427d-abad-053b4ec83c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.459856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f7c94-f656-427d-abad-053b4ec83c71 None admin] PATH_INFO: `/v3/users/d51637a5abf94eaea17f462b29cbbadf` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.461034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f7c94-f656-427d-abad-053b4ec83c71 None admin] RBAC: Authorizing `identity:get_user(user_id=d51637a5abf94eaea17f462b29cbbadf)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:36.462504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f7c94-f656-427d-abad-053b4ec83c71 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:36.464436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 166/328] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:36 2022] GET /identity/v3/users/d51637a5abf94eaea17f462b29cbbadf => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:36.469831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f7c94-f656-427d-abad-053b4ec83c71 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:36.518691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e1574-fbea-4759-b3ef-24bce2374106 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:36.549220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:36.552161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb314a9-cf17-43f7-a0de-dce5decf2dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:36.552776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb314a9-cf17-43f7-a0de-dce5decf2dde None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.553070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb314a9-cf17-43f7-a0de-dce5decf2dde None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.553340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb314a9-cf17-43f7-a0de-dce5decf2dde None admin] PATH_INFO: `/v3/groups` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.553958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb314a9-cf17-43f7-a0de-dce5decf2dde None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:36.557874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb314a9-cf17-43f7-a0de-dce5decf2dde None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:36.574166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f7c94-f656-427d-abad-053b4ec83c71 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:36.607446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8dd3a-6b1e-451c-af38-dabf0c1174b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:36.608034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8dd3a-6b1e-451c-af38-dabf0c1174b8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.608307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8dd3a-6b1e-451c-af38-dabf0c1174b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.608572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8dd3a-6b1e-451c-af38-dabf0c1174b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.609306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8dd3a-6b1e-451c-af38-dabf0c1174b8 None admin] RBAC: Adding query filter params (role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, user.id=d51637a5abf94eaea17f462b29cbbadf, scope.system=all) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:36.609610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8dd3a-6b1e-451c-af38-dabf0c1174b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:36.611026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8dd3a-6b1e-451c-af38-dabf0c1174b8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:36.622693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 167/329] 10.222.0.84 () {56 vars in 1308 bytes} [Wed Aug 24 11:11:36 2022] GET /identity/v3/role_assignments?user.id=d51637a5abf94eaea17f462b29cbbadf&scope.system=all&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 510 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:36.628421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8dd3a-6b1e-451c-af38-dabf0c1174b8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:36.696831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 163/330] 10.222.0.84 () {60 vars in 1134 bytes} [Wed Aug 24 11:11:36 2022] POST /identity/v3/groups => generated 226 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:11:36.756680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8dd3a-6b1e-451c-af38-dabf0c1174b8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:36.797091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:36.802530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53606706-7dd7-4ad8-ba66-50d8da0c824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:36.803205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53606706-7dd7-4ad8-ba66-50d8da0c824c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.803523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53606706-7dd7-4ad8-ba66-50d8da0c824c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.803819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53606706-7dd7-4ad8-ba66-50d8da0c824c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.817519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53606706-7dd7-4ad8-ba66-50d8da0c824c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-53606706-7dd7-4ad8-ba66-50d8da0c824c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:36.822645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:36.823563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:36.824475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 168/331] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:36.830107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb314a9-cf17-43f7-a0de-dce5decf2dde None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:36.929047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:36.936366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb314a9-cf17-43f7-a0de-dce5decf2dde None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:36.974750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73f933-f29f-426a-9fe3-40728cf7e6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:36.975364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73f933-f29f-426a-9fe3-40728cf7e6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:36.975637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73f933-f29f-426a-9fe3-40728cf7e6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:36.975906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73f933-f29f-426a-9fe3-40728cf7e6d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:36.976582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73f933-f29f-426a-9fe3-40728cf7e6d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:36.976960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73f933-f29f-426a-9fe3-40728cf7e6d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:36.979034 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73f933-f29f-426a-9fe3-40728cf7e6d7 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:36.989170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 164/332] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:36.995271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53606706-7dd7-4ad8-ba66-50d8da0c824c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:37.105992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53606706-7dd7-4ad8-ba66-50d8da0c824c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:37.144036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8b547-41f2-490c-8750-5273afde0e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:37.144685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8b547-41f2-490c-8750-5273afde0e4d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:37.144987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8b547-41f2-490c-8750-5273afde0e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:37.145263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8b547-41f2-490c-8750-5273afde0e4d None admin] PATH_INFO: `/v3/users/aefef4c62280482582e7804465a9a0eb` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:37.146409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8b547-41f2-490c-8750-5273afde0e4d None admin] RBAC: Authorizing `identity:get_user(user_id=aefef4c62280482582e7804465a9a0eb)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:37.147849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8b547-41f2-490c-8750-5273afde0e4d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:37.149468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 169/333] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:36 2022] GET /identity/v3/users/aefef4c62280482582e7804465a9a0eb => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:37.156379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73f933-f29f-426a-9fe3-40728cf7e6d7 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:37.269760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73f933-f29f-426a-9fe3-40728cf7e6d7 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:37.306452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab42481-c107-4b7b-b3f8-dd2f6b6812f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:37.307118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab42481-c107-4b7b-b3f8-dd2f6b6812f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:37.307392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab42481-c107-4b7b-b3f8-dd2f6b6812f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:37.307654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab42481-c107-4b7b-b3f8-dd2f6b6812f3 None admin] PATH_INFO: `/v3/system/users/aefef4c62280482582e7804465a9a0eb/roles/c54d8e4858914575b2ba91acf3233251` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:37.309132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab42481-c107-4b7b-b3f8-dd2f6b6812f3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=aefef4c62280482582e7804465a9a0eb, role_id=c54d8e4858914575b2ba91acf3233251)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:37.310506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab42481-c107-4b7b-b3f8-dd2f6b6812f3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:37.393159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 165/334] 10.222.0.84 () {58 vars in 1334 bytes} [Wed Aug 24 11:11:37 2022] PUT /identity/v3/system/users/aefef4c62280482582e7804465a9a0eb/roles/c54d8e4858914575b2ba91acf3233251 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:38.865617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d2132-7745-4745-82b9-aa744a296c87 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:38.866139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d2132-7745-4745-82b9-aa744a296c87 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:38.866139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d2132-7745-4745-82b9-aa744a296c87 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:38.867497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 170/335] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:38.873369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8aed34-2ea4-458c-9346-6981c3157875 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:38.873837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8aed34-2ea4-458c-9346-6981c3157875 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:38.873935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8aed34-2ea4-458c-9346-6981c3157875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:38.892706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-4f8aed34-2ea4-458c-9346-6981c3157875 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:38.894243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4f8aed34-2ea4-458c-9346-6981c3157875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:38.914046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:38.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 166/336] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:38.983483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d43bb-3442-4604-aec2-c22db3253a81 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:38.984041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d43bb-3442-4604-aec2-c22db3253a81 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:38.984518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d43bb-3442-4604-aec2-c22db3253a81 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:38.986075 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 171/337] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:38.991459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e4a10-8653-44af-bc13-10a4eb9e5e51 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:38.992261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e4a10-8653-44af-bc13-10a4eb9e5e51 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:38.992984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e4a10-8653-44af-bc13-10a4eb9e5e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:39.021798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a78e4a10-8653-44af-bc13-10a4eb9e5e51 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:39.023008 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a78e4a10-8653-44af-bc13-10a4eb9e5e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:39.098388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 167/338] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:39.448482 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d43bb-3442-4604-aec2-c22db3253a81 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:39.490924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e4a10-8653-44af-bc13-10a4eb9e5e51 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:39.531016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:39.595620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d43bb-3442-4604-aec2-c22db3253a81 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:39.609183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:39.640896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e4a10-8653-44af-bc13-10a4eb9e5e51 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:39.644374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634ab15-7964-4859-9478-ec3c8207c994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:39.645037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634ab15-7964-4859-9478-ec3c8207c994 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:39.645308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634ab15-7964-4859-9478-ec3c8207c994 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:39.645633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634ab15-7964-4859-9478-ec3c8207c994 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:39.672183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634ab15-7964-4859-9478-ec3c8207c994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-b634ab15-7964-4859-9478-ec3c8207c994 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:39.677413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:39.678469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:39.679429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:39.687228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 172/339] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:39.691845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634ab15-7964-4859-9478-ec3c8207c994 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:39.732500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f6b70-00c0-44b7-bc2b-a28563d45f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:39.733183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f6b70-00c0-44b7-bc2b-a28563d45f5b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:39.733469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f6b70-00c0-44b7-bc2b-a28563d45f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:39.733740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f6b70-00c0-44b7-bc2b-a28563d45f5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:39.773774 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f6b70-00c0-44b7-bc2b-a28563d45f5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-9c9f6b70-00c0-44b7-bc2b-a28563d45f5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:39.783737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:39.784652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:39.786138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 168/340] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:39.786885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e142d6-6f5e-4b1b-9bcb-feda02e890d4 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:39.786885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e142d6-6f5e-4b1b-9bcb-feda02e890d4 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:39.787025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e142d6-6f5e-4b1b-9bcb-feda02e890d4 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:39.788213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 169/341] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:39.796452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12470954-1a9f-4f33-8a3e-a8c6faab9f82 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:39.796814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12470954-1a9f-4f33-8a3e-a8c6faab9f82 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:39.797117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12470954-1a9f-4f33-8a3e-a8c6faab9f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:39.836353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-12470954-1a9f-4f33-8a3e-a8c6faab9f82 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:39.837559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-12470954-1a9f-4f33-8a3e-a8c6faab9f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:39.850952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634ab15-7964-4859-9478-ec3c8207c994 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:39.894031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:39.933155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667158b0-15c1-447a-9223-b7badd27827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:39.933872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667158b0-15c1-447a-9223-b7badd27827b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:39.934134 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667158b0-15c1-447a-9223-b7badd27827b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:39.934397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667158b0-15c1-447a-9223-b7badd27827b None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:39.935116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667158b0-15c1-447a-9223-b7badd27827b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:39.935416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667158b0-15c1-447a-9223-b7badd27827b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:39.936829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667158b0-15c1-447a-9223-b7badd27827b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:39.952250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 173/342] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:39.953112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667158b0-15c1-447a-9223-b7badd27827b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:39.969530 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 170/343] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:39.970418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12470954-1a9f-4f33-8a3e-a8c6faab9f82 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:40.136908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:40.185848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667158b0-15c1-447a-9223-b7badd27827b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:40.215405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12470954-1a9f-4f33-8a3e-a8c6faab9f82 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:40.245532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9a3e4-a516-456d-99c5-29d4b72b3eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:40.246157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9a3e4-a516-456d-99c5-29d4b72b3eda None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:40.246470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9a3e4-a516-456d-99c5-29d4b72b3eda None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:40.246812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9a3e4-a516-456d-99c5-29d4b72b3eda None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:40.247478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9a3e4-a516-456d-99c5-29d4b72b3eda None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:40.247779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9a3e4-a516-456d-99c5-29d4b72b3eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:40.249434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9a3e4-a516-456d-99c5-29d4b72b3eda None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:40.260956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 174/344] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:40.264962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9a3e4-a516-456d-99c5-29d4b72b3eda None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:40.285188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3615381-c72c-49ef-826c-da15e085b9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:40.285805 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3615381-c72c-49ef-826c-da15e085b9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:40.286082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3615381-c72c-49ef-826c-da15e085b9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:40.286353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3615381-c72c-49ef-826c-da15e085b9f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:40.329146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:40.399916 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9a3e4-a516-456d-99c5-29d4b72b3eda None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:40.451624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019493f-8ba8-4831-b928-bdefe4f684fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:40.452242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019493f-8ba8-4831-b928-bdefe4f684fb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:40.452507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019493f-8ba8-4831-b928-bdefe4f684fb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:40.452779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019493f-8ba8-4831-b928-bdefe4f684fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:40.488372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3615381-c72c-49ef-826c-da15e085b9f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:40.491036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3615381-c72c-49ef-826c-da15e085b9f1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:40.586937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019493f-8ba8-4831-b928-bdefe4f684fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:40.588436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019493f-8ba8-4831-b928-bdefe4f684fb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:40.659365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:40.663108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:40.684274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 171/345] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:40.685148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3615381-c72c-49ef-826c-da15e085b9f1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:40.723546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 175/346] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:11:40.724401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019493f-8ba8-4831-b928-bdefe4f684fb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:40.846954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3615381-c72c-49ef-826c-da15e085b9f1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:40.876463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019493f-8ba8-4831-b928-bdefe4f684fb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:40.895621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365e864-e3b0-4598-855a-6edd7a98f3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:40.896374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365e864-e3b0-4598-855a-6edd7a98f3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:40.896729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365e864-e3b0-4598-855a-6edd7a98f3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:40.897094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365e864-e3b0-4598-855a-6edd7a98f3c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:40.912187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365e864-e3b0-4598-855a-6edd7a98f3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8365e864-e3b0-4598-855a-6edd7a98f3c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:40.919186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:40.920095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 172/347] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:40.921679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365e864-e3b0-4598-855a-6edd7a98f3c6 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:40.928649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70b0cd-b94c-4897-a3bc-97e8041360ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:40.930022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70b0cd-b94c-4897-a3bc-97e8041360ff None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:40.930535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70b0cd-b94c-4897-a3bc-97e8041360ff None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:40.931572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70b0cd-b94c-4897-a3bc-97e8041360ff None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:40.932956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70b0cd-b94c-4897-a3bc-97e8041360ff None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:40.934585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70b0cd-b94c-4897-a3bc-97e8041360ff None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:40.936407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 176/348] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:40 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:40.937933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ea3b9-75a5-471b-9a0b-d6601b3b2ba6 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:40.938418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ea3b9-75a5-471b-9a0b-d6601b3b2ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:40.938884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ea3b9-75a5-471b-9a0b-d6601b3b2ba6 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:40.942418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 177/349] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:40.943338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ea3b9-75a5-471b-9a0b-d6601b3b2ba6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:40.959983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:40.966826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:41.072331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ea3b9-75a5-471b-9a0b-d6601b3b2ba6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:41.072917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365e864-e3b0-4598-855a-6edd7a98f3c6 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:41.119571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0779aa-57b0-45f0-a015-6f6741870489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:41.120217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0779aa-57b0-45f0-a015-6f6741870489 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.120495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0779aa-57b0-45f0-a015-6f6741870489 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.120762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0779aa-57b0-45f0-a015-6f6741870489 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.121985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0779aa-57b0-45f0-a015-6f6741870489 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:41.122541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b5b75-6d57-4af4-89e8-682809e591ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:41.123155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b5b75-6d57-4af4-89e8-682809e591ba None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.123432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b5b75-6d57-4af4-89e8-682809e591ba None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.123533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0779aa-57b0-45f0-a015-6f6741870489 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:41.123704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b5b75-6d57-4af4-89e8-682809e591ba None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.124365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b5b75-6d57-4af4-89e8-682809e591ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:41.124671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b5b75-6d57-4af4-89e8-682809e591ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:41.125308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 173/350] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:40 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:41.129030 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b5b75-6d57-4af4-89e8-682809e591ba None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:41.129176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ed404-8d05-469c-b6f6-392b0ff4db7e None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.129176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ed404-8d05-469c-b6f6-392b0ff4db7e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.129176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ed404-8d05-469c-b6f6-392b0ff4db7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.137518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 178/351] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:41.138543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b5b75-6d57-4af4-89e8-682809e591ba None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:41.146854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-164ed404-8d05-469c-b6f6-392b0ff4db7e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:41.148094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-164ed404-8d05-469c-b6f6-392b0ff4db7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:41.253329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:41.260823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 174/352] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:41.260823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f2187-3760-4474-8b56-95fbe35ccb62 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.260823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f2187-3760-4474-8b56-95fbe35ccb62 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.260823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f2187-3760-4474-8b56-95fbe35ccb62 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.261358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 175/353] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:41.262057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f2187-3760-4474-8b56-95fbe35ccb62 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:41.291257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b5b75-6d57-4af4-89e8-682809e591ba None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:41.302044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:41.338733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8c4a4-ecf2-4d02-8abd-722b2336cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:41.339429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8c4a4-ecf2-4d02-8abd-722b2336cd6b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.339683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8c4a4-ecf2-4d02-8abd-722b2336cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.339957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8c4a4-ecf2-4d02-8abd-722b2336cd6b None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.348839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8c4a4-ecf2-4d02-8abd-722b2336cd6b None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:41.350825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8c4a4-ecf2-4d02-8abd-722b2336cd6b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:41.353525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 179/354] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:41 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:41.357703 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8c4a4-ecf2-4d02-8abd-722b2336cd6b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:41.413907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f2187-3760-4474-8b56-95fbe35ccb62 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:41.461313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6632056-f461-47fa-9ccf-b2782c8799e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:41.462123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6632056-f461-47fa-9ccf-b2782c8799e5 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.463536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6632056-f461-47fa-9ccf-b2782c8799e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.463818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6632056-f461-47fa-9ccf-b2782c8799e5 None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.467609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6632056-f461-47fa-9ccf-b2782c8799e5 None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:41.467609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6632056-f461-47fa-9ccf-b2782c8799e5 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:41.471551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 176/355] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:41 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:41.471551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b73254-c956-4a0e-9a60-474aa43ae8e2 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.471551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b73254-c956-4a0e-9a60-474aa43ae8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.471551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b73254-c956-4a0e-9a60-474aa43ae8e2 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.472936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 177/356] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:41.474918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca529564-ac9e-4c5c-9ae3-fc3db527feb5 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.475408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca529564-ac9e-4c5c-9ae3-fc3db527feb5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.476104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca529564-ac9e-4c5c-9ae3-fc3db527feb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.489525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8c4a4-ecf2-4d02-8abd-722b2336cd6b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:41.500036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ca529564-ac9e-4c5c-9ae3-fc3db527feb5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:41.501362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ca529564-ac9e-4c5c-9ae3-fc3db527feb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:41.540053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dd8ef-6c73-4112-bfb6-88be7f50d699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:41.540970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dd8ef-6c73-4112-bfb6-88be7f50d699 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.541384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dd8ef-6c73-4112-bfb6-88be7f50d699 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.541786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dd8ef-6c73-4112-bfb6-88be7f50d699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.557615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:41.586342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 178/357] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:41.587343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca529564-ac9e-4c5c-9ae3-fc3db527feb5 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:41.619461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:41.636520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8dd8ef-6c73-4112-bfb6-88be7f50d699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:41.638148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8dd8ef-6c73-4112-bfb6-88be7f50d699 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:41.727728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca529564-ac9e-4c5c-9ae3-fc3db527feb5 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:41.774152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 180/358] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:41.775190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dd8ef-6c73-4112-bfb6-88be7f50d699 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:41.778835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bf4b2-6320-43a6-9ccb-02e4afc8e27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:41.779826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bf4b2-6320-43a6-9ccb-02e4afc8e27a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.780292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bf4b2-6320-43a6-9ccb-02e4afc8e27a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.780770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bf4b2-6320-43a6-9ccb-02e4afc8e27a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.781874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bf4b2-6320-43a6-9ccb-02e4afc8e27a None admin] RBAC: Adding query filter params (group.id=ba86f17371584e8cba75d2e0494c1f9e, role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:41.782201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bf4b2-6320-43a6-9ccb-02e4afc8e27a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:41.783627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bf4b2-6320-43a6-9ccb-02e4afc8e27a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:41.791574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 179/359] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:41 2022] GET /identity/v3/role_assignments?group.id=ba86f17371584e8cba75d2e0494c1f9e&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:41.792942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7debe7-59e3-4b6c-ad26-e6fc53a324ed None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.793224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7debe7-59e3-4b6c-ad26-e6fc53a324ed None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.793486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7debe7-59e3-4b6c-ad26-e6fc53a324ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.814214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6b7debe7-59e3-4b6c-ad26-e6fc53a324ed None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:41.815414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6b7debe7-59e3-4b6c-ad26-e6fc53a324ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:41.825117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:41.908444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 180/360] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:41.909342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7debe7-59e3-4b6c-ad26-e6fc53a324ed None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:41.917179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dd8ef-6c73-4112-bfb6-88be7f50d699 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:41.938918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:41.959158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea365-1c7e-4d2c-95a7-754d5add5bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:41.960569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea365-1c7e-4d2c-95a7-754d5add5bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:41.961243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea365-1c7e-4d2c-95a7-754d5add5bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:41.961824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea365-1c7e-4d2c-95a7-754d5add5bd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:41.962832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ea365-1c7e-4d2c-95a7-754d5add5bd6 None admin] RBAC: Adding query filter params (group.id=ba86f17371584e8cba75d2e0494c1f9e, role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:41.963543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ea365-1c7e-4d2c-95a7-754d5add5bd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:41.966641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ea365-1c7e-4d2c-95a7-754d5add5bd6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:41.975576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 181/361] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:41 2022] GET /identity/v3/role_assignments?group.id=ba86f17371584e8cba75d2e0494c1f9e&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:41.976582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea365-1c7e-4d2c-95a7-754d5add5bd6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:42.065532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7debe7-59e3-4b6c-ad26-e6fc53a324ed None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.135798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa641fc-4955-4511-abac-66615c2357ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:42.139813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa641fc-4955-4511-abac-66615c2357ce None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:42.140073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa641fc-4955-4511-abac-66615c2357ce None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:42.140338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa641fc-4955-4511-abac-66615c2357ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:42.161569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa641fc-4955-4511-abac-66615c2357ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-7fa641fc-4955-4511-abac-66615c2357ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:42.167836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:42.168750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 11:11:42.169562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:42.170152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea365-1c7e-4d2c-95a7-754d5add5bd6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.170152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:42.175832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 181/362] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:42.175832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa641fc-4955-4511-abac-66615c2357ce None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:42.235860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ab2fe-a222-427d-a33c-56da69afb0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:42.236498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab2fe-a222-427d-a33c-56da69afb0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:42.236776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab2fe-a222-427d-a33c-56da69afb0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:42.237087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab2fe-a222-427d-a33c-56da69afb0d8 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:42.238387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ab2fe-a222-427d-a33c-56da69afb0d8 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:42.240657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ab2fe-a222-427d-a33c-56da69afb0d8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:42.242388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 182/363] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:41 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:42.243149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ab2fe-a222-427d-a33c-56da69afb0d8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:42.305401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:42.344597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa641fc-4955-4511-abac-66615c2357ce None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.384008 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abc80d-fea8-4679-9884-767d52672804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:42.384997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abc80d-fea8-4679-9884-767d52672804 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:42.385276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abc80d-fea8-4679-9884-767d52672804 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:42.385554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abc80d-fea8-4679-9884-767d52672804 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:42.386603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ab2fe-a222-427d-a33c-56da69afb0d8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.402867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abc80d-fea8-4679-9884-767d52672804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-79abc80d-fea8-4679-9884-767d52672804 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:42.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:42.410395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:42.411182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:42.411782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 182/364] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:42.412840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abc80d-fea8-4679-9884-767d52672804 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:42.428272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a3101-01fc-4249-89f5-c68b0bac067a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:42.429182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a3101-01fc-4249-89f5-c68b0bac067a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:42.429457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a3101-01fc-4249-89f5-c68b0bac067a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:42.429727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a3101-01fc-4249-89f5-c68b0bac067a None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:42.430388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a3101-01fc-4249-89f5-c68b0bac067a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:42.430687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a3101-01fc-4249-89f5-c68b0bac067a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:42.432161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a3101-01fc-4249-89f5-c68b0bac067a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:42.440770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 183/365] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:42.441589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a3101-01fc-4249-89f5-c68b0bac067a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:42.486709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:42.550536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abc80d-fea8-4679-9884-767d52672804 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.578064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a3101-01fc-4249-89f5-c68b0bac067a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.597935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca54a9e-77f4-4ff6-a216-53f150dfc777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:42.598114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca54a9e-77f4-4ff6-a216-53f150dfc777 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:42.598318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca54a9e-77f4-4ff6-a216-53f150dfc777 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:42.598567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca54a9e-77f4-4ff6-a216-53f150dfc777 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:42.611313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:42.615037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca54a9e-77f4-4ff6-a216-53f150dfc777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-6ca54a9e-77f4-4ff6-a216-53f150dfc777 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:42.623495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:42.643309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:42.644063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 183/366] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:11:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:42.644688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15226a97-e5c0-4078-94b8-a55a5598b9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:42.644688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15226a97-e5c0-4078-94b8-a55a5598b9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:42.644688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15226a97-e5c0-4078-94b8-a55a5598b9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:42.644688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15226a97-e5c0-4078-94b8-a55a5598b9c6 None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:42.644947 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca54a9e-77f4-4ff6-a216-53f150dfc777 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:42.645024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15226a97-e5c0-4078-94b8-a55a5598b9c6 None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:42.645024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15226a97-e5c0-4078-94b8-a55a5598b9c6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:42.645024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 184/367] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:42 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:42.645024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15226a97-e5c0-4078-94b8-a55a5598b9c6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:42.760176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca54a9e-77f4-4ff6-a216-53f150dfc777 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.777832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:42.782585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15226a97-e5c0-4078-94b8-a55a5598b9c6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.803634 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0682478-b506-4dd0-b1f5-52ff7a16bb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:42.804262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0682478-b506-4dd0-b1f5-52ff7a16bb06 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:42.804540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0682478-b506-4dd0-b1f5-52ff7a16bb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:42.804838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0682478-b506-4dd0-b1f5-52ff7a16bb06 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:42.805508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0682478-b506-4dd0-b1f5-52ff7a16bb06 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:42.805809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0682478-b506-4dd0-b1f5-52ff7a16bb06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:42.809036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0682478-b506-4dd0-b1f5-52ff7a16bb06 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:42.818707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 184/368] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:42.819823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0682478-b506-4dd0-b1f5-52ff7a16bb06 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:42.827193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c452b63-cb9c-46a6-a8c1-49bf3c69020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:42.828104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c452b63-cb9c-46a6-a8c1-49bf3c69020b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:42.828575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c452b63-cb9c-46a6-a8c1-49bf3c69020b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:42.829098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c452b63-cb9c-46a6-a8c1-49bf3c69020b None admin] PATH_INFO: `/v3/users/aefef4c62280482582e7804465a9a0eb` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:42.831171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c452b63-cb9c-46a6-a8c1-49bf3c69020b None admin] RBAC: Authorizing `identity:get_user(user_id=aefef4c62280482582e7804465a9a0eb)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:42.833153 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c452b63-cb9c-46a6-a8c1-49bf3c69020b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:42.835071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 185/369] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:11:42 2022] GET /identity/v3/users/aefef4c62280482582e7804465a9a0eb => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:42.836089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c452b63-cb9c-46a6-a8c1-49bf3c69020b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:42.903269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:42.947235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0682478-b506-4dd0-b1f5-52ff7a16bb06 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.960421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c452b63-cb9c-46a6-a8c1-49bf3c69020b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:42.994531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2e00d-2acd-430e-aa11-c8bf6e10d396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:42.995146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2e00d-2acd-430e-aa11-c8bf6e10d396 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:42.995469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2e00d-2acd-430e-aa11-c8bf6e10d396 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:42.995742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2e00d-2acd-430e-aa11-c8bf6e10d396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:42.996671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2e00d-2acd-430e-aa11-c8bf6e10d396 None admin] RBAC: Adding query filter params (group.id=ba86f17371584e8cba75d2e0494c1f9e, role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=931647de90cc4037a03cac17bd65f53d) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:42.996964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2e00d-2acd-430e-aa11-c8bf6e10d396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:42.998551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2e00d-2acd-430e-aa11-c8bf6e10d396 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:43.005349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 185/370] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:42 2022] GET /identity/v3/role_assignments?group.id=ba86f17371584e8cba75d2e0494c1f9e&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:43.007685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfa751-5a3f-4a6e-980a-fd4cabdcb794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:43.008361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfa751-5a3f-4a6e-980a-fd4cabdcb794 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:43.008644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfa751-5a3f-4a6e-980a-fd4cabdcb794 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:43.008754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2e00d-2acd-430e-aa11-c8bf6e10d396 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:43.008947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfa751-5a3f-4a6e-980a-fd4cabdcb794 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:43.009719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfa751-5a3f-4a6e-980a-fd4cabdcb794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:43.010030 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfa751-5a3f-4a6e-980a-fd4cabdcb794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:43.011441 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfa751-5a3f-4a6e-980a-fd4cabdcb794 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:43.030232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 186/371] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:43.031253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfa751-5a3f-4a6e-980a-fd4cabdcb794 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:43.092498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:43.212722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2e00d-2acd-430e-aa11-c8bf6e10d396 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:43.229535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfa751-5a3f-4a6e-980a-fd4cabdcb794 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:43.250275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:43.263832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d43bc-e9c0-4f4d-802f-d62505e59741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:43.266576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d43bc-e9c0-4f4d-802f-d62505e59741 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:43.266576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d43bc-e9c0-4f4d-802f-d62505e59741 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:43.266858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d43bc-e9c0-4f4d-802f-d62505e59741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:43.303981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743e0b0-aa09-418d-a08f-9450b5913905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:43.304602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e0b0-aa09-418d-a08f-9450b5913905 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:43.304902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e0b0-aa09-418d-a08f-9450b5913905 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:43.305177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743e0b0-aa09-418d-a08f-9450b5913905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:43.305900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743e0b0-aa09-418d-a08f-9450b5913905 None admin] RBAC: Adding query filter params (role.id=c54d8e4858914575b2ba91acf3233251, user.id=aefef4c62280482582e7804465a9a0eb, scope.system=all) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:43.306319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743e0b0-aa09-418d-a08f-9450b5913905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:43.307750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743e0b0-aa09-418d-a08f-9450b5913905 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:43.334669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 187/372] 10.222.0.84 () {56 vars in 1308 bytes} [Wed Aug 24 11:11:43 2022] GET /identity/v3/role_assignments?user.id=aefef4c62280482582e7804465a9a0eb&scope.system=all&role.id=c54d8e4858914575b2ba91acf3233251 => generated 510 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:43.335524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743e0b0-aa09-418d-a08f-9450b5913905 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:43.425088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d43bc-e9c0-4f4d-802f-d62505e59741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:43.427567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d43bc-e9c0-4f4d-802f-d62505e59741 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:43.517272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:43.531659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743e0b0-aa09-418d-a08f-9450b5913905 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:43.572616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc944e1-ede4-4b70-b3f8-fe4cd4dbcf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:43.573273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc944e1-ede4-4b70-b3f8-fe4cd4dbcf67 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:43.573549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc944e1-ede4-4b70-b3f8-fe4cd4dbcf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:43.573822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc944e1-ede4-4b70-b3f8-fe4cd4dbcf67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:43.643599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 186/373] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:43.653860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d43bc-e9c0-4f4d-802f-d62505e59741 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:43.679256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc944e1-ede4-4b70-b3f8-fe4cd4dbcf67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:43.682062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc944e1-ede4-4b70-b3f8-fe4cd4dbcf67 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:43.687225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:43.777473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d43bc-e9c0-4f4d-802f-d62505e59741 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:43.825111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ac31b-0d78-4a44-9161-9775e2982c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:43.828867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ac31b-0d78-4a44-9161-9775e2982c3d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:43.828867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ac31b-0d78-4a44-9161-9775e2982c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:43.828867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ac31b-0d78-4a44-9161-9775e2982c3d None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:43.828867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ac31b-0d78-4a44-9161-9775e2982c3d None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:43.829802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ac31b-0d78-4a44-9161-9775e2982c3d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:43.832062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 187/374] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:43 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:43.838788 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ac31b-0d78-4a44-9161-9775e2982c3d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:43.860889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:43.891016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 188/375] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:43.901955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc944e1-ede4-4b70-b3f8-fe4cd4dbcf67 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:43.967964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:43.975512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ac31b-0d78-4a44-9161-9775e2982c3d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:44.023332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3881906-730b-45dd-8dd2-440346ac8422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:44.024024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3881906-730b-45dd-8dd2-440346ac8422 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:44.024305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3881906-730b-45dd-8dd2-440346ac8422 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:44.024571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3881906-730b-45dd-8dd2-440346ac8422 None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:44.028884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3881906-730b-45dd-8dd2-440346ac8422 None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:44.028884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3881906-730b-45dd-8dd2-440346ac8422 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:44.030488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 188/376] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:43 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:44.037537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3881906-730b-45dd-8dd2-440346ac8422 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:44.162563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc944e1-ede4-4b70-b3f8-fe4cd4dbcf67 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:44.198670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3881906-730b-45dd-8dd2-440346ac8422 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:44.234014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c5865-fc23-4b28-81cf-bd95b2509716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:44.234889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c5865-fc23-4b28-81cf-bd95b2509716 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:44.235171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c5865-fc23-4b28-81cf-bd95b2509716 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:44.235444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c5865-fc23-4b28-81cf-bd95b2509716 None admin] PATH_INFO: `/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:44.237567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c5865-fc23-4b28-81cf-bd95b2509716 None admin] RBAC: Authorizing `identity:get_project(project_id=cd4c9f105cf4428d9f0cb67e8376cb7e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:44.239037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c5865-fc23-4b28-81cf-bd95b2509716 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:44.241024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 189/377] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:43 2022] GET /identity/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e => generated 344 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:44.250000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c5865-fc23-4b28-81cf-bd95b2509716 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:44.270566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe0f3c-6d9b-427e-ba30-e38fa587f111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:44.271219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0f3c-6d9b-427e-ba30-e38fa587f111 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:44.271731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0f3c-6d9b-427e-ba30-e38fa587f111 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:44.272014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0f3c-6d9b-427e-ba30-e38fa587f111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:44.272749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe0f3c-6d9b-427e-ba30-e38fa587f111 None admin] RBAC: Adding query filter params (group.id=ba86f17371584e8cba75d2e0494c1f9e, role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=931647de90cc4037a03cac17bd65f53d) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:44.273352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe0f3c-6d9b-427e-ba30-e38fa587f111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:44.277628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe0f3c-6d9b-427e-ba30-e38fa587f111 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:44.288527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 189/378] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:44 2022] GET /identity/v3/role_assignments?group.id=ba86f17371584e8cba75d2e0494c1f9e&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:44.344962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:44.362509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b3f86-b972-4737-927f-c7cbd78249d2 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:44.362920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b3f86-b972-4737-927f-c7cbd78249d2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:44.363321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b3f86-b972-4737-927f-c7cbd78249d2 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:44.365770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 190/379] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:11:44.373001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634280d-fe70-4066-b9ba-31ddaf77b223 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:44.373416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634280d-fe70-4066-b9ba-31ddaf77b223 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:44.373817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634280d-fe70-4066-b9ba-31ddaf77b223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:44.408823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-0634280d-fe70-4066-b9ba-31ddaf77b223 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:44.409646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0634280d-fe70-4066-b9ba-31ddaf77b223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:44.427802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c5865-fc23-4b28-81cf-bd95b2509716 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:44.471580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0398b9-17fc-4eb0-a9c0-be07ee4f8d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:44.472691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0398b9-17fc-4eb0-a9c0-be07ee4f8d12 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:44.473226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:44.474047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0398b9-17fc-4eb0-a9c0-be07ee4f8d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:44.474522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0398b9-17fc-4eb0-a9c0-be07ee4f8d12 None admin] PATH_INFO: `/v3/groups/3fc3f46946864359b73c4a65bab5c3d4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:44.482038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0398b9-17fc-4eb0-a9c0-be07ee4f8d12 None admin] RBAC: Authorizing `identity:get_group(group_id=3fc3f46946864359b73c4a65bab5c3d4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:44.483901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0398b9-17fc-4eb0-a9c0-be07ee4f8d12 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:44.486194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 190/380] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:44 2022] GET /identity/v3/groups/3fc3f46946864359b73c4a65bab5c3d4 => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:44.494578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0398b9-17fc-4eb0-a9c0-be07ee4f8d12 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:44.576448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 191/381] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:44.578150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf6145-a405-4f09-a9f1-1849fcb3b4ad None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:44.579703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf6145-a405-4f09-a9f1-1849fcb3b4ad None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:44.580014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf6145-a405-4f09-a9f1-1849fcb3b4ad None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:44.581828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 192/382] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:44.590300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7aa36-1961-4aca-a91e-e8214c54b528 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:44.590585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7aa36-1961-4aca-a91e-e8214c54b528 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:44.590851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7aa36-1961-4aca-a91e-e8214c54b528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:44.637860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c0e7aa36-1961-4aca-a91e-e8214c54b528 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:44.639068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c0e7aa36-1961-4aca-a91e-e8214c54b528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:44.663523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0398b9-17fc-4eb0-a9c0-be07ee4f8d12 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:44.686816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:44.703587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7e6cc-bcad-4f3a-b376-572a050cc7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:44.704217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7e6cc-bcad-4f3a-b376-572a050cc7fd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:44.704493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7e6cc-bcad-4f3a-b376-572a050cc7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:44.704765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7e6cc-bcad-4f3a-b376-572a050cc7fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:44.707132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7e6cc-bcad-4f3a-b376-572a050cc7fd None admin] RBAC: Adding query filter params (group.id=3fc3f46946864359b73c4a65bab5c3d4, role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=cd4c9f105cf4428d9f0cb67e8376cb7e) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:44.707132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7e6cc-bcad-4f3a-b376-572a050cc7fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:44.707399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7e6cc-bcad-4f3a-b376-572a050cc7fd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:44.715157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 191/383] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:44 2022] GET /identity/v3/role_assignments?group.id=3fc3f46946864359b73c4a65bab5c3d4&scope.project.id=cd4c9f105cf4428d9f0cb67e8376cb7e&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:44.802514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 193/384] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:44.954058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7e6cc-bcad-4f3a-b376-572a050cc7fd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:45.091768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7e6cc-bcad-4f3a-b376-572a050cc7fd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:45.129897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7aa36-1961-4aca-a91e-e8214c54b528 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:45.135006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091a222-2c25-4d20-b33d-b49fbfcdd3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:45.135006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091a222-2c25-4d20-b33d-b49fbfcdd3fa None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.135006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091a222-2c25-4d20-b33d-b49fbfcdd3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.135006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091a222-2c25-4d20-b33d-b49fbfcdd3fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.147537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091a222-2c25-4d20-b33d-b49fbfcdd3fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c091a222-2c25-4d20-b33d-b49fbfcdd3fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:45.154088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:45.154959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:45.155712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:45.158305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 192/385] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:45.169794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091a222-2c25-4d20-b33d-b49fbfcdd3fa None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:45.239418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:45.293698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091a222-2c25-4d20-b33d-b49fbfcdd3fa None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:45.330366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfc224-9056-4a61-803c-b2fb9c90308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:45.331001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfc224-9056-4a61-803c-b2fb9c90308e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.331279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfc224-9056-4a61-803c-b2fb9c90308e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.331552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfc224-9056-4a61-803c-b2fb9c90308e None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.332223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfc224-9056-4a61-803c-b2fb9c90308e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:45.332528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfc224-9056-4a61-803c-b2fb9c90308e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:45.334465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfc224-9056-4a61-803c-b2fb9c90308e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:45.344698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 193/386] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:45.350603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfc224-9056-4a61-803c-b2fb9c90308e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:45.396356 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:45.448844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7aa36-1961-4aca-a91e-e8214c54b528 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:45.467420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfc224-9056-4a61-803c-b2fb9c90308e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:45.479070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:45.504566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622cf7a-5c9e-4fc1-868b-a86be9ed2c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:45.505158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622cf7a-5c9e-4fc1-868b-a86be9ed2c33 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.505580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622cf7a-5c9e-4fc1-868b-a86be9ed2c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.505851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622cf7a-5c9e-4fc1-868b-a86be9ed2c33 None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.507287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622cf7a-5c9e-4fc1-868b-a86be9ed2c33 None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:45.508734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622cf7a-5c9e-4fc1-868b-a86be9ed2c33 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:45.511242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 194/387] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:45 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:45.517757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622cf7a-5c9e-4fc1-868b-a86be9ed2c33 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:45.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48f726-75a4-4f7c-aca2-f1eee62be053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:45.566277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48f726-75a4-4f7c-aca2-f1eee62be053 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.566555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48f726-75a4-4f7c-aca2-f1eee62be053 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.566821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48f726-75a4-4f7c-aca2-f1eee62be053 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.621506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48f726-75a4-4f7c-aca2-f1eee62be053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-1d48f726-75a4-4f7c-aca2-f1eee62be053 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:45.626266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:45.627111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:45.627921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 194/388] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 498 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:45.629489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622cf7a-5c9e-4fc1-868b-a86be9ed2c33 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:45.637794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48f726-75a4-4f7c-aca2-f1eee62be053 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:45.680598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5137b44-8216-41df-9bbc-c4c418a5d7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:45.681820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5137b44-8216-41df-9bbc-c4c418a5d7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.682112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5137b44-8216-41df-9bbc-c4c418a5d7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.682451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5137b44-8216-41df-9bbc-c4c418a5d7e1 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.683948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5137b44-8216-41df-9bbc-c4c418a5d7e1 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:45.685454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5137b44-8216-41df-9bbc-c4c418a5d7e1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:45.687396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 195/389] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:45 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:45.693049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5137b44-8216-41df-9bbc-c4c418a5d7e1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:45.753166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:45.820200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48f726-75a4-4f7c-aca2-f1eee62be053 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:45.825036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5137b44-8216-41df-9bbc-c4c418a5d7e1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:45.865323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea06411-8230-488d-a03f-e54b126205a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:45.866497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea06411-8230-488d-a03f-e54b126205a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.866919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea06411-8230-488d-a03f-e54b126205a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.867334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea06411-8230-488d-a03f-e54b126205a5 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/groups/ba86f17371584e8cba75d2e0494c1f9e/roles/1fbe1ab7a2864ba1818c7038baf57c34` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.869376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f6173-d044-410a-9d39-bf780dbe90a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:45.870114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f6173-d044-410a-9d39-bf780dbe90a6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.870392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f6173-d044-410a-9d39-bf780dbe90a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.870660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f6173-d044-410a-9d39-bf780dbe90a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.871045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea06411-8230-488d-a03f-e54b126205a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2eee6585d6f04caf9d4244aa86bb7c0c, group_id=ba86f17371584e8cba75d2e0494c1f9e, role_id=1fbe1ab7a2864ba1818c7038baf57c34)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:45.871325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f6173-d044-410a-9d39-bf780dbe90a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:45.871624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f6173-d044-410a-9d39-bf780dbe90a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:45.873823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f6173-d044-410a-9d39-bf780dbe90a6 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:45.874198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea06411-8230-488d-a03f-e54b126205a5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:45.882680 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 195/390] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:45.883714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66ebe3-1ed1-4015-86b5-3cb977199704 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.884023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66ebe3-1ed1-4015-86b5-3cb977199704 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.884294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66ebe3-1ed1-4015-86b5-3cb977199704 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:45.888897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 196/391] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:45.888897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66ebe3-1ed1-4015-86b5-3cb977199704 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:45.902825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:45.975149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 196/392] 10.222.0.84 () {58 vars in 1442 bytes} [Wed Aug 24 11:11:45 2022] PUT /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/groups/ba86f17371584e8cba75d2e0494c1f9e/roles/1fbe1ab7a2864ba1818c7038baf57c34 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:45.976671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a6296-552a-419b-bc6a-b11d65c663e4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:45.977166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a6296-552a-419b-bc6a-b11d65c663e4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:45.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a6296-552a-419b-bc6a-b11d65c663e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:46.008899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ba4a6296-552a-419b-bc6a-b11d65c663e4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:46.013868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ba4a6296-552a-419b-bc6a-b11d65c663e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:46.048950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66ebe3-1ed1-4015-86b5-3cb977199704 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:46.122677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c5283-f747-41ce-a589-32db3cc63500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:46.123514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c5283-f747-41ce-a589-32db3cc63500 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:46.123885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c5283-f747-41ce-a589-32db3cc63500 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:46.124248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c5283-f747-41ce-a589-32db3cc63500 None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:46.125510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c5283-f747-41ce-a589-32db3cc63500 None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:46.127160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c5283-f747-41ce-a589-32db3cc63500 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:46.129201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 197/393] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:45 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:46.135277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c5283-f747-41ce-a589-32db3cc63500 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:46.145256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 197/394] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:46.250544 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:46.277175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c5283-f747-41ce-a589-32db3cc63500 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:46.315806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebba1d3-405a-4214-b37c-dbb505e0f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:46.316446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebba1d3-405a-4214-b37c-dbb505e0f88f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:46.316723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebba1d3-405a-4214-b37c-dbb505e0f88f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:46.317038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebba1d3-405a-4214-b37c-dbb505e0f88f None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:46.318193 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebba1d3-405a-4214-b37c-dbb505e0f88f None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:46.319637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebba1d3-405a-4214-b37c-dbb505e0f88f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:46.321655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 198/395] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:46 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:46.330140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a6296-552a-419b-bc6a-b11d65c663e4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:46.377955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:46.447413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebba1d3-405a-4214-b37c-dbb505e0f88f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:46.470839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a6296-552a-419b-bc6a-b11d65c663e4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:46.514337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12633b62-472c-44cf-b4ae-c51b2230c5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:46.515386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12633b62-472c-44cf-b4ae-c51b2230c5ee None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:46.515861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12633b62-472c-44cf-b4ae-c51b2230c5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:46.516352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12633b62-472c-44cf-b4ae-c51b2230c5ee None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/groups/ba86f17371584e8cba75d2e0494c1f9e/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:46.518510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12633b62-472c-44cf-b4ae-c51b2230c5ee None admin] RBAC: Authorizing `identity:create_grant(project_id=2eee6585d6f04caf9d4244aa86bb7c0c, group_id=ba86f17371584e8cba75d2e0494c1f9e, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:46.520143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12633b62-472c-44cf-b4ae-c51b2230c5ee None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:46.586908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebba1d3-405a-4214-b37c-dbb505e0f88f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:46.609822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 198/396] 10.222.0.84 () {58 vars in 1442 bytes} [Wed Aug 24 11:11:46 2022] PUT /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c/groups/ba86f17371584e8cba75d2e0494c1f9e/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:46.644320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3f82e-d2a1-486e-9112-cb0c545375bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:46.644993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f82e-d2a1-486e-9112-cb0c545375bf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:46.645325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f82e-d2a1-486e-9112-cb0c545375bf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:46.645600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3f82e-d2a1-486e-9112-cb0c545375bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:46.663748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3f82e-d2a1-486e-9112-cb0c545375bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-6ab3f82e-d2a1-486e-9112-cb0c545375bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:46.669367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:46.670294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:46.671117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 199/397] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:11:46.675098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a81c-ce06-455b-9865-331d229b2dd5 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:46.675683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a81c-ce06-455b-9865-331d229b2dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:46.676155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a81c-ce06-455b-9865-331d229b2dd5 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:46.678239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 199/398] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:46.684277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120605f9-2268-4a22-afd9-de2108e9be20 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:46.684579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120605f9-2268-4a22-afd9-de2108e9be20 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:46.684850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0a81c-ce06-455b-9865-331d229b2dd5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:46.684953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120605f9-2268-4a22-afd9-de2108e9be20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:46.709997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:46.719437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-120605f9-2268-4a22-afd9-de2108e9be20 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:46.720658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-120605f9-2268-4a22-afd9-de2108e9be20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:46.827273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:46.835165 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 200/399] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:46.841474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0a81c-ce06-455b-9865-331d229b2dd5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:46.885141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993b281-9eae-4bb0-b0fa-2b4e6c483009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:46.886001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993b281-9eae-4bb0-b0fa-2b4e6c483009 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:46.886422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993b281-9eae-4bb0-b0fa-2b4e6c483009 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:46.886862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993b281-9eae-4bb0-b0fa-2b4e6c483009 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:46.887756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993b281-9eae-4bb0-b0fa-2b4e6c483009 None admin] RBAC: Adding query filter params (name=member) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:46.888204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993b281-9eae-4bb0-b0fa-2b4e6c483009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:46.894300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993b281-9eae-4bb0-b0fa-2b4e6c483009 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:46.904459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 200/400] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:46.910068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120605f9-2268-4a22-afd9-de2108e9be20 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:47.040868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120605f9-2268-4a22-afd9-de2108e9be20 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:47.101757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ed3b5-8889-45f1-8779-98ffe95c0b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:47.102700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ed3b5-8889-45f1-8779-98ffe95c0b27 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.103120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ed3b5-8889-45f1-8779-98ffe95c0b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.103523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ed3b5-8889-45f1-8779-98ffe95c0b27 None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:47.107671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ed3b5-8889-45f1-8779-98ffe95c0b27 None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:47.109230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ed3b5-8889-45f1-8779-98ffe95c0b27 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:47.111583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 201/401] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:46 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:47.117344 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993b281-9eae-4bb0-b0fa-2b4e6c483009 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:47.260257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993b281-9eae-4bb0-b0fa-2b4e6c483009 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:47.272005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ed3b5-8889-45f1-8779-98ffe95c0b27 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:47.275487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:47.313205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a3843-a768-4409-a4ab-daaa2e05330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:47.314207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a3843-a768-4409-a4ab-daaa2e05330b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.314710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a3843-a768-4409-a4ab-daaa2e05330b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.315179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a3843-a768-4409-a4ab-daaa2e05330b None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:47.316661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a3843-a768-4409-a4ab-daaa2e05330b None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:47.320180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a3843-a768-4409-a4ab-daaa2e05330b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:47.322881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 201/402] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:47 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:47.330192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a3843-a768-4409-a4ab-daaa2e05330b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:47.373219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:47.460341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ed3b5-8889-45f1-8779-98ffe95c0b27 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:47.489665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a3843-a768-4409-a4ab-daaa2e05330b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:47.506007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24445e4c-396b-4df6-8703-85f7bbc399c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:47.507217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24445e4c-396b-4df6-8703-85f7bbc399c1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.507744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24445e4c-396b-4df6-8703-85f7bbc399c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.508368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24445e4c-396b-4df6-8703-85f7bbc399c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:47.522931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24445e4c-396b-4df6-8703-85f7bbc399c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:47.527345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d69c26-5889-4dee-a06c-db15cae71a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:47.528203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d69c26-5889-4dee-a06c-db15cae71a2d None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.528484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d69c26-5889-4dee-a06c-db15cae71a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.528756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d69c26-5889-4dee-a06c-db15cae71a2d None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d/groups/ba86f17371584e8cba75d2e0494c1f9e/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-24445e4c-396b-4df6-8703-85f7bbc399c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:47.529436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:47.530381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:47.531155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:47.531747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d69c26-5889-4dee-a06c-db15cae71a2d None admin] RBAC: Authorizing `identity:create_grant(project_id=931647de90cc4037a03cac17bd65f53d, group_id=ba86f17371584e8cba75d2e0494c1f9e, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:47.532293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d69c26-5889-4dee-a06c-db15cae71a2d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:47.533924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 202/403] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:47.538737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b5714-8850-428e-a8d1-e4090be11c47 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.538737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b5714-8850-428e-a8d1-e4090be11c47 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.538737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b5714-8850-428e-a8d1-e4090be11c47 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:47.541267 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 203/404] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:47.541983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b5714-8850-428e-a8d1-e4090be11c47 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:47.614036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:47.663102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 202/405] 10.222.0.84 () {58 vars in 1442 bytes} [Wed Aug 24 11:11:47 2022] PUT /identity/v3/projects/931647de90cc4037a03cac17bd65f53d/groups/ba86f17371584e8cba75d2e0494c1f9e/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:47.664105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55555122-7626-40ee-a978-f79f01a68019 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.664386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55555122-7626-40ee-a978-f79f01a68019 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.664655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55555122-7626-40ee-a978-f79f01a68019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:47.669177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b5714-8850-428e-a8d1-e4090be11c47 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:47.696391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-55555122-7626-40ee-a978-f79f01a68019 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:47.699495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-55555122-7626-40ee-a978-f79f01a68019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:47.765893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e1cb5-62b9-4d4b-91c4-1e230b4a615e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:47.767584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e1cb5-62b9-4d4b-91c4-1e230b4a615e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.767584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e1cb5-62b9-4d4b-91c4-1e230b4a615e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.767786 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e1cb5-62b9-4d4b-91c4-1e230b4a615e None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:47.768611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e1cb5-62b9-4d4b-91c4-1e230b4a615e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:47.768936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e1cb5-62b9-4d4b-91c4-1e230b4a615e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:47.770508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e1cb5-62b9-4d4b-91c4-1e230b4a615e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:47.788266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 204/406] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:47.794377 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e1cb5-62b9-4d4b-91c4-1e230b4a615e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:47.798530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 203/407] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:47.918934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e1cb5-62b9-4d4b-91c4-1e230b4a615e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:47.945076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:47.957953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63c06c-54e6-41c4-a86b-1a09122a9f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:47.958649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63c06c-54e6-41c4-a86b-1a09122a9f7b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:47.958980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63c06c-54e6-41c4-a86b-1a09122a9f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:47.959255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63c06c-54e6-41c4-a86b-1a09122a9f7b None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:47.960520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63c06c-54e6-41c4-a86b-1a09122a9f7b None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:47.961995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63c06c-54e6-41c4-a86b-1a09122a9f7b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:47.964083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 205/408] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:47 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:47.969632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55555122-7626-40ee-a978-f79f01a68019 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:47.992731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:48.108250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55555122-7626-40ee-a978-f79f01a68019 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.157965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23129b-965b-4aea-a947-3738b1236b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.158600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23129b-965b-4aea-a947-3738b1236b69 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.158875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23129b-965b-4aea-a947-3738b1236b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.159145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23129b-965b-4aea-a947-3738b1236b69 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.160809 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23129b-965b-4aea-a947-3738b1236b69 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.162275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23129b-965b-4aea-a947-3738b1236b69 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.164214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 204/409] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:47 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:48.170489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63c06c-54e6-41c4-a86b-1a09122a9f7b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.226203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23129b-965b-4aea-a947-3738b1236b69 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.313345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63c06c-54e6-41c4-a86b-1a09122a9f7b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.348881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:48.351024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8ca0b-c9d9-4f42-aaac-5776e69a9cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.351918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8ca0b-c9d9-4f42-aaac-5776e69a9cb3 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.352360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8ca0b-c9d9-4f42-aaac-5776e69a9cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.353494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8ca0b-c9d9-4f42-aaac-5776e69a9cb3 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d/groups/ba86f17371584e8cba75d2e0494c1f9e/roles/1fbe1ab7a2864ba1818c7038baf57c34` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.355855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8ca0b-c9d9-4f42-aaac-5776e69a9cb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=931647de90cc4037a03cac17bd65f53d, group_id=ba86f17371584e8cba75d2e0494c1f9e, role_id=1fbe1ab7a2864ba1818c7038baf57c34)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.359516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8ca0b-c9d9-4f42-aaac-5776e69a9cb3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.370513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23129b-965b-4aea-a947-3738b1236b69 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.406271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd0c4a-d5b9-41a7-8655-c30d73c61f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.406923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd0c4a-d5b9-41a7-8655-c30d73c61f14 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.407166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd0c4a-d5b9-41a7-8655-c30d73c61f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.407431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd0c4a-d5b9-41a7-8655-c30d73c61f14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.418756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd0c4a-d5b9-41a7-8655-c30d73c61f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-0abd0c4a-d5b9-41a7-8655-c30d73c61f14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:48.423456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:48.424348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:48.425575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 205/410] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:11:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:11:48.426259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bd52-ded9-4521-b1ff-4fbfb0695885 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.426571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bd52-ded9-4521-b1ff-4fbfb0695885 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.426665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94bd52-ded9-4521-b1ff-4fbfb0695885 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.432357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 206/411] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:48.432948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94bd52-ded9-4521-b1ff-4fbfb0695885 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.453204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 206/412] 10.222.0.84 () {58 vars in 1442 bytes} [Wed Aug 24 11:11:48 2022] PUT /identity/v3/projects/931647de90cc4037a03cac17bd65f53d/groups/ba86f17371584e8cba75d2e0494c1f9e/roles/1fbe1ab7a2864ba1818c7038baf57c34 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:48.454964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accffef6-60f0-43e0-a1fd-d0fbfa59486d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.455245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accffef6-60f0-43e0-a1fd-d0fbfa59486d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.455561 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accffef6-60f0-43e0-a1fd-d0fbfa59486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.480331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-accffef6-60f0-43e0-a1fd-d0fbfa59486d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:48.482435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-accffef6-60f0-43e0-a1fd-d0fbfa59486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:48.548152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:48.576454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 207/413] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:48.657106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94bd52-ded9-4521-b1ff-4fbfb0695885 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.704442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44228552-a83e-419d-b632-d85e54ddd090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.705082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44228552-a83e-419d-b632-d85e54ddd090 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.705358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44228552-a83e-419d-b632-d85e54ddd090 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.705629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44228552-a83e-419d-b632-d85e54ddd090 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.706295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44228552-a83e-419d-b632-d85e54ddd090 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:48.706593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44228552-a83e-419d-b632-d85e54ddd090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.707970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44228552-a83e-419d-b632-d85e54ddd090 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.717511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 207/414] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:48.722919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accffef6-60f0-43e0-a1fd-d0fbfa59486d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.865259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accffef6-60f0-43e0-a1fd-d0fbfa59486d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:48.880978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44228552-a83e-419d-b632-d85e54ddd090 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:48.884456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:48.927505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e232f-0b24-41bb-aeaa-ff5e5dc262c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:48.928066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e232f-0b24-41bb-aeaa-ff5e5dc262c1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:48.928335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e232f-0b24-41bb-aeaa-ff5e5dc262c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:48.928601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e232f-0b24-41bb-aeaa-ff5e5dc262c1 None admin] PATH_INFO: `/v3/groups/3fc3f46946864359b73c4a65bab5c3d4` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:48.930451 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e232f-0b24-41bb-aeaa-ff5e5dc262c1 None admin] RBAC: Authorizing `identity:get_group(group_id=3fc3f46946864359b73c4a65bab5c3d4)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:48.931962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e232f-0b24-41bb-aeaa-ff5e5dc262c1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:48.934796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 208/415] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:48 2022] GET /identity/v3/groups/3fc3f46946864359b73c4a65bab5c3d4 => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:48.941768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e232f-0b24-41bb-aeaa-ff5e5dc262c1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.023655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:49.048745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44228552-a83e-419d-b632-d85e54ddd090 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.080350 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e232f-0b24-41bb-aeaa-ff5e5dc262c1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.093315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193ad14-92c1-4789-847b-469383b63a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.093948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193ad14-92c1-4789-847b-469383b63a72 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.094220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193ad14-92c1-4789-847b-469383b63a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.094485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193ad14-92c1-4789-847b-469383b63a72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.109103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193ad14-92c1-4789-847b-469383b63a72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e193ad14-92c1-4789-847b-469383b63a72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:49.116266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:49.117197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:49.118041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 208/416] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:49.119146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10781b-5596-4f62-acb4-7ae822033684 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.119554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10781b-5596-4f62-acb4-7ae822033684 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.119831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10781b-5596-4f62-acb4-7ae822033684 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.124028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56888fcc-31ca-410f-a669-b2a250ab82fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.124964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56888fcc-31ca-410f-a669-b2a250ab82fb None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.125258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 209/417] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:49.125699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56888fcc-31ca-410f-a669-b2a250ab82fb None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.126027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10781b-5596-4f62-acb4-7ae822033684 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.126760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56888fcc-31ca-410f-a669-b2a250ab82fb None admin] PATH_INFO: `/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.128097 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56888fcc-31ca-410f-a669-b2a250ab82fb None admin] RBAC: Authorizing `identity:get_project(project_id=cd4c9f105cf4428d9f0cb67e8376cb7e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:49.130036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56888fcc-31ca-410f-a669-b2a250ab82fb None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:49.132229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 209/418] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:48 2022] GET /identity/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:49.138345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760b517-39b4-4991-a11e-d8d8eb82558b None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.139130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760b517-39b4-4991-a11e-d8d8eb82558b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.139701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760b517-39b4-4991-a11e-d8d8eb82558b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.161557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9760b517-39b4-4991-a11e-d8d8eb82558b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:49.163147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9760b517-39b4-4991-a11e-d8d8eb82558b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:49.198626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:49.243173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10781b-5596-4f62-acb4-7ae822033684 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.248885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 210/419] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:49.250592 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760b517-39b4-4991-a11e-d8d8eb82558b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.309771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5699b9-3c65-4c67-a5da-959edb3b85ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.310688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5699b9-3c65-4c67-a5da-959edb3b85ac None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.311941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5699b9-3c65-4c67-a5da-959edb3b85ac None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.312357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5699b9-3c65-4c67-a5da-959edb3b85ac None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.314398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5699b9-3c65-4c67-a5da-959edb3b85ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:49.314860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5699b9-3c65-4c67-a5da-959edb3b85ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:49.317188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5699b9-3c65-4c67-a5da-959edb3b85ac None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:49.330248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 210/420] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:49.335205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5699b9-3c65-4c67-a5da-959edb3b85ac None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.352384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:49.439622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760b517-39b4-4991-a11e-d8d8eb82558b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.450789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5699b9-3c65-4c67-a5da-959edb3b85ac None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.471179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:49.496342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711109b3-7560-42f7-bbc2-12761a6d7b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.497605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711109b3-7560-42f7-bbc2-12761a6d7b12 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.497744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398f1ea-dfca-4850-bb12-ced9f30e91fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.498213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711109b3-7560-42f7-bbc2-12761a6d7b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.498324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398f1ea-dfca-4850-bb12-ced9f30e91fc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.498751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711109b3-7560-42f7-bbc2-12761a6d7b12 None admin] PATH_INFO: `/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e/groups/3fc3f46946864359b73c4a65bab5c3d4/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.498867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398f1ea-dfca-4850-bb12-ced9f30e91fc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.499083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398f1ea-dfca-4850-bb12-ced9f30e91fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.501409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711109b3-7560-42f7-bbc2-12761a6d7b12 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd4c9f105cf4428d9f0cb67e8376cb7e, group_id=3fc3f46946864359b73c4a65bab5c3d4, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:49.503154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711109b3-7560-42f7-bbc2-12761a6d7b12 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:49.580296 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 211/421] 10.222.0.84 () {58 vars in 1442 bytes} [Wed Aug 24 11:11:49 2022] PUT /identity/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e/groups/3fc3f46946864359b73c4a65bab5c3d4/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:11:49.590248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398f1ea-dfca-4850-bb12-ced9f30e91fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:49.592004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398f1ea-dfca-4850-bb12-ced9f30e91fc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:49.638696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711109b3-7560-42f7-bbc2-12761a6d7b12 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.799045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 211/422] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:49.804276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398f1ea-dfca-4850-bb12-ced9f30e91fc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.816388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711109b3-7560-42f7-bbc2-12761a6d7b12 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.827158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:49.839847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:49.861645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb53ee-8181-46b0-8b38-a9910ef065f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.862567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb53ee-8181-46b0-8b38-a9910ef065f2 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.864539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb53ee-8181-46b0-8b38-a9910ef065f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.865391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb53ee-8181-46b0-8b38-a9910ef065f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.879432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb53ee-8181-46b0-8b38-a9910ef065f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-85bb53ee-8181-46b0-8b38-a9910ef065f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:49.886169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:49.887058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:49.887954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:49.890702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 212/423] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:49.896318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb53ee-8181-46b0-8b38-a9910ef065f2 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:49.937706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398f1ea-dfca-4850-bb12-ced9f30e91fc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:49.976608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa57b9b-6b02-4866-a986-c2d1283b1774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:49.977379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa57b9b-6b02-4866-a986-c2d1283b1774 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.977724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa57b9b-6b02-4866-a986-c2d1283b1774 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.978551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa57b9b-6b02-4866-a986-c2d1283b1774 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.979745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa57b9b-6b02-4866-a986-c2d1283b1774 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:49.981814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa57b9b-6b02-4866-a986-c2d1283b1774 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:49.989346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 212/424] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:49 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:49.989346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80316e8b-ace5-485a-bea7-56380f88e901 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:49.989346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80316e8b-ace5-485a-bea7-56380f88e901 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:49.989346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80316e8b-ace5-485a-bea7-56380f88e901 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:49.989346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 213/425] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:49.989346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80316e8b-ace5-485a-bea7-56380f88e901 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:50.039916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb53ee-8181-46b0-8b38-a9910ef065f2 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:50.085901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026e71e-10a7-43a3-bafd-0ad6eba0fa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:50.087693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e71e-10a7-43a3-bafd-0ad6eba0fa0c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:50.088194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e71e-10a7-43a3-bafd-0ad6eba0fa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:50.088625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026e71e-10a7-43a3-bafd-0ad6eba0fa0c None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:50.089470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026e71e-10a7-43a3-bafd-0ad6eba0fa0c None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:50.089914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026e71e-10a7-43a3-bafd-0ad6eba0fa0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:50.092334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026e71e-10a7-43a3-bafd-0ad6eba0fa0c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:50.094392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:50.110487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 213/426] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:50.110692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80316e8b-ace5-485a-bea7-56380f88e901 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:50.110787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abebf4d-8568-47b2-8601-c019cf34905c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:50.110787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abebf4d-8568-47b2-8601-c019cf34905c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:50.110787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abebf4d-8568-47b2-8601-c019cf34905c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:50.141564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9abebf4d-8568-47b2-8601-c019cf34905c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:50.143131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9abebf4d-8568-47b2-8601-c019cf34905c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:50.160211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fab491-1ce9-49d9-8075-c16fc8fba406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:50.160866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fab491-1ce9-49d9-8075-c16fc8fba406 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:50.161313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fab491-1ce9-49d9-8075-c16fc8fba406 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:50.161593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fab491-1ce9-49d9-8075-c16fc8fba406 None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:50.162769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fab491-1ce9-49d9-8075-c16fc8fba406 None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:50.164644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fab491-1ce9-49d9-8075-c16fc8fba406 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:50.172066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 214/427] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:49 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:50.172066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fab491-1ce9-49d9-8075-c16fc8fba406 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:50.182172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:50.238594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 214/428] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:50.239909 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abebf4d-8568-47b2-8601-c019cf34905c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:50.342954 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fab491-1ce9-49d9-8075-c16fc8fba406 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:50.354708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abebf4d-8568-47b2-8601-c019cf34905c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:50.393766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0dda5f-7c95-4618-84aa-071eafcf654e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:50.394734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dda5f-7c95-4618-84aa-071eafcf654e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:50.395224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dda5f-7c95-4618-84aa-071eafcf654e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:50.395688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0dda5f-7c95-4618-84aa-071eafcf654e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:50.396609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0dda5f-7c95-4618-84aa-071eafcf654e None admin] RBAC: Adding query filter params (group.id=ba86f17371584e8cba75d2e0494c1f9e, role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:50.397126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0dda5f-7c95-4618-84aa-071eafcf654e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:50.399212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0dda5f-7c95-4618-84aa-071eafcf654e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:50.406608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 215/429] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:50 2022] GET /identity/v3/role_assignments?group.id=ba86f17371584e8cba75d2e0494c1f9e&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:50.466691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa275e7b-b3dd-40be-bcaf-0a21f241ac45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:50.467379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa275e7b-b3dd-40be-bcaf-0a21f241ac45 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:50.467588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa275e7b-b3dd-40be-bcaf-0a21f241ac45 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:50.467856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa275e7b-b3dd-40be-bcaf-0a21f241ac45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:50.523843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:50.581798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa275e7b-b3dd-40be-bcaf-0a21f241ac45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:50.583267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa275e7b-b3dd-40be-bcaf-0a21f241ac45 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:50.588881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0dda5f-7c95-4618-84aa-071eafcf654e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:50.629812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:50.715065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 215/430] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:50.716330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9966b-d94b-4c08-a8b5-16e6ea9b8c42 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:50.716616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9966b-d94b-4c08-a8b5-16e6ea9b8c42 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:50.716915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9966b-d94b-4c08-a8b5-16e6ea9b8c42 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:50.718257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 216/431] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:50.720412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9966b-d94b-4c08-a8b5-16e6ea9b8c42 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:50.730436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0dda5f-7c95-4618-84aa-071eafcf654e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:50.765676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81cf78-9ec2-45df-ac20-abe29e075e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:50.766305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81cf78-9ec2-45df-ac20-abe29e075e83 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:50.766581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81cf78-9ec2-45df-ac20-abe29e075e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:50.766849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81cf78-9ec2-45df-ac20-abe29e075e83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:50.777689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81cf78-9ec2-45df-ac20-abe29e075e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-af81cf78-9ec2-45df-ac20-abe29e075e83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:50.782860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:50.783726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:50.784506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 216/432] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:11:50.785188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6ed4a-a91b-4724-ad4f-dfb0fb50589e None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:50.785738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6ed4a-a91b-4724-ad4f-dfb0fb50589e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:50.785738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6ed4a-a91b-4724-ad4f-dfb0fb50589e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:50.796694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:50.803946 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-96b6ed4a-a91b-4724-ad4f-dfb0fb50589e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:50.805155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-96b6ed4a-a91b-4724-ad4f-dfb0fb50589e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:50.844389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9966b-d94b-4c08-a8b5-16e6ea9b8c42 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:50.877857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 217/433] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:50.881848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6ed4a-a91b-4724-ad4f-dfb0fb50589e None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:50.892650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a411f0e-a27a-4f0b-856e-994e0b59b834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:50.893672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a411f0e-a27a-4f0b-856e-994e0b59b834 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:50.894220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a411f0e-a27a-4f0b-856e-994e0b59b834 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:50.894687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a411f0e-a27a-4f0b-856e-994e0b59b834 None admin] PATH_INFO: `/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:50.896164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a411f0e-a27a-4f0b-856e-994e0b59b834 None admin] RBAC: Authorizing `identity:get_project(project_id=2eee6585d6f04caf9d4244aa86bb7c0c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:50.897449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:50.898940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a411f0e-a27a-4f0b-856e-994e0b59b834 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:50.901108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 217/434] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:50 2022] GET /identity/v3/projects/2eee6585d6f04caf9d4244aa86bb7c0c => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:50.906847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a411f0e-a27a-4f0b-856e-994e0b59b834 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:50.994272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6ed4a-a91b-4724-ad4f-dfb0fb50589e None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:51.036956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74baad-52ef-44b1-9887-0e74ab4d4b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:51.037831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74baad-52ef-44b1-9887-0e74ab4d4b89 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:51.038115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74baad-52ef-44b1-9887-0e74ab4d4b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:51.038391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74baad-52ef-44b1-9887-0e74ab4d4b89 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:51.039052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74baad-52ef-44b1-9887-0e74ab4d4b89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:51.039371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74baad-52ef-44b1-9887-0e74ab4d4b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:51.042347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74baad-52ef-44b1-9887-0e74ab4d4b89 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:51.044289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a411f0e-a27a-4f0b-856e-994e0b59b834 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:51.052776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 218/435] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:11:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:51.062499 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74baad-52ef-44b1-9887-0e74ab4d4b89 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:51.076068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:51.085069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98260a55-b8b3-4a69-8fe5-d46f2bddf0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:51.087371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98260a55-b8b3-4a69-8fe5-d46f2bddf0ca None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:51.087371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98260a55-b8b3-4a69-8fe5-d46f2bddf0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:51.087371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98260a55-b8b3-4a69-8fe5-d46f2bddf0ca None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:51.088126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98260a55-b8b3-4a69-8fe5-d46f2bddf0ca None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:51.089597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98260a55-b8b3-4a69-8fe5-d46f2bddf0ca None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:51.091153 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 218/436] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:50 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:51.098049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98260a55-b8b3-4a69-8fe5-d46f2bddf0ca None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:51.169770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:51.194237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74baad-52ef-44b1-9887-0e74ab4d4b89 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:51.214636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98260a55-b8b3-4a69-8fe5-d46f2bddf0ca None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:51.229718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece761-d649-4265-bd21-3f9bbcf79cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:51.230413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece761-d649-4265-bd21-3f9bbcf79cbc None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:51.230688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece761-d649-4265-bd21-3f9bbcf79cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:51.230960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ece761-d649-4265-bd21-3f9bbcf79cbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:51.249339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576862d-dba0-45b2-8d35-e327d07da255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:51.249933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576862d-dba0-45b2-8d35-e327d07da255 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:51.250205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576862d-dba0-45b2-8d35-e327d07da255 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:51.250473 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576862d-dba0-45b2-8d35-e327d07da255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:51.251256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576862d-dba0-45b2-8d35-e327d07da255 None admin] RBAC: Adding query filter params (group.id=ba86f17371584e8cba75d2e0494c1f9e, role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:51.251561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576862d-dba0-45b2-8d35-e327d07da255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:51.253009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576862d-dba0-45b2-8d35-e327d07da255 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:51.258635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 219/437] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:51 2022] GET /identity/v3/role_assignments?group.id=ba86f17371584e8cba75d2e0494c1f9e&scope.project.id=2eee6585d6f04caf9d4244aa86bb7c0c&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:51.259552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576862d-dba0-45b2-8d35-e327d07da255 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:51.327926 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:51.335943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ece761-d649-4265-bd21-3f9bbcf79cbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:51.338239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ece761-d649-4265-bd21-3f9bbcf79cbc None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:51.388419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576862d-dba0-45b2-8d35-e327d07da255 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:51.426651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:51.438148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bf805-5a21-42a7-a3d5-c1fa184d2efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:51.439167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf805-5a21-42a7-a3d5-c1fa184d2efa None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:51.439167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf805-5a21-42a7-a3d5-c1fa184d2efa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:51.453258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf805-5a21-42a7-a3d5-c1fa184d2efa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:51.458176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bf805-5a21-42a7-a3d5-c1fa184d2efa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:51.458444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 219/438] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e36bf805-5a21-42a7-a3d5-c1fa184d2efa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:51.464053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:51.465029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:51.465806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:51.466394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 220/439] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:51.467889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece761-d649-4265-bd21-3f9bbcf79cbc None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:51.471633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bf805-5a21-42a7-a3d5-c1fa184d2efa None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:51.600527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ece761-d649-4265-bd21-3f9bbcf79cbc None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:51.620417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bf805-5a21-42a7-a3d5-c1fa184d2efa None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:51.647610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:51.658850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3639c1-4455-415b-8117-bccf6f2af410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:51.665933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3639c1-4455-415b-8117-bccf6f2af410 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:51.665933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3639c1-4455-415b-8117-bccf6f2af410 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:51.665933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3639c1-4455-415b-8117-bccf6f2af410 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:51.665933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3639c1-4455-415b-8117-bccf6f2af410 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:51.665933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3639c1-4455-415b-8117-bccf6f2af410 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:51.665933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 220/440] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:51 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:51.669578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba5b6-7daa-4254-8f2f-72b07cd34f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:51.670420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba5b6-7daa-4254-8f2f-72b07cd34f34 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:51.670856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba5b6-7daa-4254-8f2f-72b07cd34f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:51.671250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba5b6-7daa-4254-8f2f-72b07cd34f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:51.671421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3639c1-4455-415b-8117-bccf6f2af410 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:51.672183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fba5b6-7daa-4254-8f2f-72b07cd34f34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:51.672620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fba5b6-7daa-4254-8f2f-72b07cd34f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:51.674337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fba5b6-7daa-4254-8f2f-72b07cd34f34 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:51.686983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 221/441] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:51.693539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba5b6-7daa-4254-8f2f-72b07cd34f34 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:51.740694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:51.815147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3639c1-4455-415b-8117-bccf6f2af410 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:51.821506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba5b6-7daa-4254-8f2f-72b07cd34f34 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:51.862793 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5888a8-def9-4241-b68e-b0655ce61018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:51.863424 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5888a8-def9-4241-b68e-b0655ce61018 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:51.863704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5888a8-def9-4241-b68e-b0655ce61018 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:51.863974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5888a8-def9-4241-b68e-b0655ce61018 None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:51.865179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5888a8-def9-4241-b68e-b0655ce61018 None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:51.866907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5888a8-def9-4241-b68e-b0655ce61018 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:51.868928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 221/442] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:51 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:51.874605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5888a8-def9-4241-b68e-b0655ce61018 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:51.876463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051577-3ebb-402e-8b42-adb092341592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:51.876463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051577-3ebb-402e-8b42-adb092341592 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:51.876463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051577-3ebb-402e-8b42-adb092341592 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:51.876731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da051577-3ebb-402e-8b42-adb092341592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:51.942068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:51.960650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da051577-3ebb-402e-8b42-adb092341592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:51.962751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da051577-3ebb-402e-8b42-adb092341592 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:52.003161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5888a8-def9-4241-b68e-b0655ce61018 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:52.015871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:52.036516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a2e12-ee68-4d47-93c2-eb752e412e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:52.037141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a2e12-ee68-4d47-93c2-eb752e412e61 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.037427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a2e12-ee68-4d47-93c2-eb752e412e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.037700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a2e12-ee68-4d47-93c2-eb752e412e61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.038416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a2e12-ee68-4d47-93c2-eb752e412e61 None admin] RBAC: Adding query filter params (group.id=ba86f17371584e8cba75d2e0494c1f9e, role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18, scope.project.id=931647de90cc4037a03cac17bd65f53d) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:52.038720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a2e12-ee68-4d47-93c2-eb752e412e61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:52.040218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a2e12-ee68-4d47-93c2-eb752e412e61 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:52.047128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 222/443] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:51 2022] GET /identity/v3/role_assignments?group.id=ba86f17371584e8cba75d2e0494c1f9e&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:52.047128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf56b-d33f-4e96-bff3-178a8c154645 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.047128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf56b-d33f-4e96-bff3-178a8c154645 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.047128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bf56b-d33f-4e96-bff3-178a8c154645 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.050185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 223/444] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:52.056342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcec8c7-f41b-4826-8a2b-4c138bf386ef None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.056623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcec8c7-f41b-4826-8a2b-4c138bf386ef None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.057088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcec8c7-f41b-4826-8a2b-4c138bf386ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.072309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 222/445] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:52.073871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051577-3ebb-402e-8b42-adb092341592 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:52.077021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ebcec8c7-f41b-4826-8a2b-4c138bf386ef None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:52.078256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ebcec8c7-f41b-4826-8a2b-4c138bf386ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:52.153636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 224/446] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:11:52.183960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:52.206672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da051577-3ebb-402e-8b42-adb092341592 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:52.247357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5e9fe-4522-47a0-bad9-0f1943142597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:52.248290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5e9fe-4522-47a0-bad9-0f1943142597 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.248290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5e9fe-4522-47a0-bad9-0f1943142597 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.248540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5e9fe-4522-47a0-bad9-0f1943142597 None admin] PATH_INFO: `/v3/projects/931647de90cc4037a03cac17bd65f53d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.249676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5e9fe-4522-47a0-bad9-0f1943142597 None admin] RBAC: Authorizing `identity:get_project(project_id=931647de90cc4037a03cac17bd65f53d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:52.251116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5e9fe-4522-47a0-bad9-0f1943142597 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:52.252740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 223/447] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:52 2022] GET /identity/v3/projects/931647de90cc4037a03cac17bd65f53d => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:52.258291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcec8c7-f41b-4826-8a2b-4c138bf386ef None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:52.364931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcec8c7-f41b-4826-8a2b-4c138bf386ef None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:52.373586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:52.401755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121d477-963d-44c4-99e3-0b88bec399cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:52.402475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121d477-963d-44c4-99e3-0b88bec399cf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.402755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121d477-963d-44c4-99e3-0b88bec399cf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.403025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121d477-963d-44c4-99e3-0b88bec399cf None admin] PATH_INFO: `/v3/groups/ba86f17371584e8cba75d2e0494c1f9e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.404129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121d477-963d-44c4-99e3-0b88bec399cf None admin] RBAC: Authorizing `identity:get_group(group_id=ba86f17371584e8cba75d2e0494c1f9e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:52.405600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121d477-963d-44c4-99e3-0b88bec399cf None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:52.407479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 225/448] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:52 2022] GET /identity/v3/groups/ba86f17371584e8cba75d2e0494c1f9e => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:52.413263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5e9fe-4522-47a0-bad9-0f1943142597 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:52.428253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121d477-963d-44c4-99e3-0b88bec399cf None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:52.522409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5e9fe-4522-47a0-bad9-0f1943142597 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:52.531524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121d477-963d-44c4-99e3-0b88bec399cf None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:52.556559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482806d-ec27-4185-b5e0-9666c6d12165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:52.557273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482806d-ec27-4185-b5e0-9666c6d12165 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.557548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482806d-ec27-4185-b5e0-9666c6d12165 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.557816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482806d-ec27-4185-b5e0-9666c6d12165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.558526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482806d-ec27-4185-b5e0-9666c6d12165 None admin] RBAC: Adding query filter params (group.id=ba86f17371584e8cba75d2e0494c1f9e, role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=931647de90cc4037a03cac17bd65f53d) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:52.558826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482806d-ec27-4185-b5e0-9666c6d12165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:52.560240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482806d-ec27-4185-b5e0-9666c6d12165 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:52.565878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 224/449] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:52 2022] GET /identity/v3/role_assignments?group.id=ba86f17371584e8cba75d2e0494c1f9e&scope.project.id=931647de90cc4037a03cac17bd65f53d&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:52.566371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dbe41-3fa7-48ac-aad8-0fb599506341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:52.566986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dbe41-3fa7-48ac-aad8-0fb599506341 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.567259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dbe41-3fa7-48ac-aad8-0fb599506341 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.567523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dbe41-3fa7-48ac-aad8-0fb599506341 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.579712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dbe41-3fa7-48ac-aad8-0fb599506341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-de6dbe41-3fa7-48ac-aad8-0fb599506341 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:52.589119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:52.590130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:52.590906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 226/450] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:11:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:11:52.596859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482806d-ec27-4185-b5e0-9666c6d12165 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:52.616499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:52.705772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482806d-ec27-4185-b5e0-9666c6d12165 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:52.742278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713aadc0-94ca-47f1-934b-ddb8503957e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:52.742886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713aadc0-94ca-47f1-934b-ddb8503957e5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.743180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713aadc0-94ca-47f1-934b-ddb8503957e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.744033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713aadc0-94ca-47f1-934b-ddb8503957e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.744707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713aadc0-94ca-47f1-934b-ddb8503957e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:52.745049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713aadc0-94ca-47f1-934b-ddb8503957e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:52.746447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713aadc0-94ca-47f1-934b-ddb8503957e5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:52.754806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 225/451] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:11:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:52.761121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dbe41-3fa7-48ac-aad8-0fb599506341 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:52.797508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:52.864873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dbe41-3fa7-48ac-aad8-0fb599506341 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:52.899846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26583d-8e93-41e7-bc90-c3256ccf8b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:52.900450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26583d-8e93-41e7-bc90-c3256ccf8b38 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:52.900720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26583d-8e93-41e7-bc90-c3256ccf8b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:52.901011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26583d-8e93-41e7-bc90-c3256ccf8b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:52.965906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26583d-8e93-41e7-bc90-c3256ccf8b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:52.967334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26583d-8e93-41e7-bc90-c3256ccf8b38 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:52.998524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:53.061810 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 227/452] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:11:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:11:53.067616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713aadc0-94ca-47f1-934b-ddb8503957e5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:53.142624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:53.164655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713aadc0-94ca-47f1-934b-ddb8503957e5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:53.198238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3aa1e-ab6d-4f2a-9c1a-c7299bd29120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:53.198849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3aa1e-ab6d-4f2a-9c1a-c7299bd29120 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:53.199126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3aa1e-ab6d-4f2a-9c1a-c7299bd29120 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:53.199394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3aa1e-ab6d-4f2a-9c1a-c7299bd29120 None admin] PATH_INFO: `/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:53.200491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3aa1e-ab6d-4f2a-9c1a-c7299bd29120 None admin] RBAC: Authorizing `identity:get_project(project_id=cd4c9f105cf4428d9f0cb67e8376cb7e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:53.201939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3aa1e-ab6d-4f2a-9c1a-c7299bd29120 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:53.203540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 226/453] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:11:53 2022] GET /identity/v3/projects/cd4c9f105cf4428d9f0cb67e8376cb7e => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:53.209036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26583d-8e93-41e7-bc90-c3256ccf8b38 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:53.305039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26583d-8e93-41e7-bc90-c3256ccf8b38 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:53.337817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e06da-148f-42f4-b81b-a4a982696d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:53.338361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e06da-148f-42f4-b81b-a4a982696d7c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:53.338637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e06da-148f-42f4-b81b-a4a982696d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:53.338904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e06da-148f-42f4-b81b-a4a982696d7c None admin] PATH_INFO: `/v3/groups/3fc3f46946864359b73c4a65bab5c3d4` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:53.339985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e06da-148f-42f4-b81b-a4a982696d7c None admin] RBAC: Authorizing `identity:get_group(group_id=3fc3f46946864359b73c4a65bab5c3d4)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:53.341451 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e06da-148f-42f4-b81b-a4a982696d7c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:53.342986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 228/454] 10.222.0.84 () {56 vars in 1180 bytes} [Wed Aug 24 11:11:53 2022] GET /identity/v3/groups/3fc3f46946864359b73c4a65bab5c3d4 => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:53.348754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3aa1e-ab6d-4f2a-9c1a-c7299bd29120 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:53.444427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3aa1e-ab6d-4f2a-9c1a-c7299bd29120 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:53.477351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245931c-3ad0-4a5f-8f74-49a0b58cbe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:53.478069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245931c-3ad0-4a5f-8f74-49a0b58cbe5a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:53.478346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245931c-3ad0-4a5f-8f74-49a0b58cbe5a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:53.478615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245931c-3ad0-4a5f-8f74-49a0b58cbe5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:53.479322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245931c-3ad0-4a5f-8f74-49a0b58cbe5a None admin] RBAC: Adding query filter params (group.id=3fc3f46946864359b73c4a65bab5c3d4, role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=cd4c9f105cf4428d9f0cb67e8376cb7e) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:53.479626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245931c-3ad0-4a5f-8f74-49a0b58cbe5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:53.481068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245931c-3ad0-4a5f-8f74-49a0b58cbe5a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:53.486878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 227/455] 10.222.0.84 () {56 vars in 1376 bytes} [Wed Aug 24 11:11:53 2022] GET /identity/v3/role_assignments?group.id=3fc3f46946864359b73c4a65bab5c3d4&scope.project.id=cd4c9f105cf4428d9f0cb67e8376cb7e&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:56.015268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e336f-18fb-42dd-b337-766c4732f46c None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.016311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e336f-18fb-42dd-b337-766c4732f46c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.016824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e336f-18fb-42dd-b337-766c4732f46c None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.018625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 229/456] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:56.024262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7913fb4-9324-423a-88c9-aed7008f3864 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.025725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7913fb4-9324-423a-88c9-aed7008f3864 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.026258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7913fb4-9324-423a-88c9-aed7008f3864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.048581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c7913fb4-9324-423a-88c9-aed7008f3864 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:56.050119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c7913fb4-9324-423a-88c9-aed7008f3864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:56.052358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:56.062807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc7ad9-7ff1-442f-a178-da9ab88c4238 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.063379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc7ad9-7ff1-442f-a178-da9ab88c4238 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.066761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc7ad9-7ff1-442f-a178-da9ab88c4238 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.066761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 230/457] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:56.071712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9bd0e-8e5b-4f7d-80d3-baf4c699c800 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.072232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9bd0e-8e5b-4f7d-80d3-baf4c699c800 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.072703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9bd0e-8e5b-4f7d-80d3-baf4c699c800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.079248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:56.095937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-81a9bd0e-8e5b-4f7d-80d3-baf4c699c800 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:56.097878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-81a9bd0e-8e5b-4f7d-80d3-baf4c699c800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:56.151883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 228/458] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:56.153164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb746-1e67-4299-a55d-0a16ef156b11 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.153447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb746-1e67-4299-a55d-0a16ef156b11 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.153720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb746-1e67-4299-a55d-0a16ef156b11 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.155180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 229/459] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:56.160339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab02b25-88b8-4c60-adcf-ddd365cd0335 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.160616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab02b25-88b8-4c60-adcf-ddd365cd0335 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.160918 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab02b25-88b8-4c60-adcf-ddd365cd0335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.193697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8ab02b25-88b8-4c60-adcf-ddd365cd0335 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:56.194877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8ab02b25-88b8-4c60-adcf-ddd365cd0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:56.201805 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 231/460] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:56.300311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 230/461] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:56.478565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9bd0e-8e5b-4f7d-80d3-baf4c699c800 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:56.527104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab02b25-88b8-4c60-adcf-ddd365cd0335 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:56.587532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9bd0e-8e5b-4f7d-80d3-baf4c699c800 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:56.628046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a06df6-26b1-440d-9ce7-2fa04468791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:56.628674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a06df6-26b1-440d-9ce7-2fa04468791e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.628988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a06df6-26b1-440d-9ce7-2fa04468791e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.629259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a06df6-26b1-440d-9ce7-2fa04468791e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.637396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:56.638661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:56.643759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a06df6-26b1-440d-9ce7-2fa04468791e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:56.646416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a06df6-26b1-440d-9ce7-2fa04468791e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-30a06df6-26b1-440d-9ce7-2fa04468791e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:11:56.685728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:56.686925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:11:56.688241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:11:56.688241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:11:56.688241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:56.688712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab02b25-88b8-4c60-adcf-ddd365cd0335 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:56.689393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 232/462] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:56.694037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a06df6-26b1-440d-9ce7-2fa04468791e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:56.740396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73154dc4-09cd-497c-8257-48c1cd1234d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:56.741328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73154dc4-09cd-497c-8257-48c1cd1234d7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.742561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73154dc4-09cd-497c-8257-48c1cd1234d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.743537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73154dc4-09cd-497c-8257-48c1cd1234d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.756962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73154dc4-09cd-497c-8257-48c1cd1234d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:56.760050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73154dc4-09cd-497c-8257-48c1cd1234d7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-73154dc4-09cd-497c-8257-48c1cd1234d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:11:56.779239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:56.780208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:11:56.781101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:11:56.781101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:11:56.781101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:56.781101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 231/463] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:56.781327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73154dc4-09cd-497c-8257-48c1cd1234d7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:56.830886 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a06df6-26b1-440d-9ce7-2fa04468791e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:56.877640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1590-a19b-43f8-98ed-c511a93e5837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:56.878195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1590-a19b-43f8-98ed-c511a93e5837 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.878476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1590-a19b-43f8-98ed-c511a93e5837 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.878749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1590-a19b-43f8-98ed-c511a93e5837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.893515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f1590-a19b-43f8-98ed-c511a93e5837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:56.895016 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f1590-a19b-43f8-98ed-c511a93e5837 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-ce6f1590-a19b-43f8-98ed-c511a93e5837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:11:56.913431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:56.914352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:11:56.915822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:11:56.915822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:11:56.915822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:56.917715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 233/464] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:56.918825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1590-a19b-43f8-98ed-c511a93e5837 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:56.919573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73154dc4-09cd-497c-8257-48c1cd1234d7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:56.941282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:56.956375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7a773-a174-4c44-bd76-3ed94c2edc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:56.957339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7a773-a174-4c44-bd76-3ed94c2edc2e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:56.957730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7a773-a174-4c44-bd76-3ed94c2edc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:56.958007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7a773-a174-4c44-bd76-3ed94c2edc2e None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:56.958649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7a773-a174-4c44-bd76-3ed94c2edc2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:56.958947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7a773-a174-4c44-bd76-3ed94c2edc2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:56.959400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:56.960305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7a773-a174-4c44-bd76-3ed94c2edc2e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:56.979052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 232/465] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:56.979052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7a773-a174-4c44-bd76-3ed94c2edc2e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:57.051004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1590-a19b-43f8-98ed-c511a93e5837 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:57.090113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3f759-3b78-4a56-90d6-3b817f1ce585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:57.090713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3f759-3b78-4a56-90d6-3b817f1ce585 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:57.090989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3f759-3b78-4a56-90d6-3b817f1ce585 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:57.091260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3f759-3b78-4a56-90d6-3b817f1ce585 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:57.091913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3f759-3b78-4a56-90d6-3b817f1ce585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:57.092209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3f759-3b78-4a56-90d6-3b817f1ce585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:57.093608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3f759-3b78-4a56-90d6-3b817f1ce585 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:57.095632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7a773-a174-4c44-bd76-3ed94c2edc2e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:57.107002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 234/466] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:57.108238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff5d2f-bd1f-491e-9c29-b1ce3199d917 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:57.108518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff5d2f-bd1f-491e-9c29-b1ce3199d917 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:57.108783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff5d2f-bd1f-491e-9c29-b1ce3199d917 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:57.114154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 235/467] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:57.114893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff5d2f-bd1f-491e-9c29-b1ce3199d917 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:57.137889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4917321-5d2e-4993-90eb-9755ed777bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:57.138777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4917321-5d2e-4993-90eb-9755ed777bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:57.139243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4917321-5d2e-4993-90eb-9755ed777bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:57.139715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4917321-5d2e-4993-90eb-9755ed777bc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:57.140427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4917321-5d2e-4993-90eb-9755ed777bc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:57.140728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4917321-5d2e-4993-90eb-9755ed777bc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:57.142197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4917321-5d2e-4993-90eb-9755ed777bc6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:57.158640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 233/468] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:57.158640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4917321-5d2e-4993-90eb-9755ed777bc6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:57.199148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:57.221456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:57.228442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff5d2f-bd1f-491e-9c29-b1ce3199d917 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:57.263168 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e371d-c767-476d-9e3d-9f62285b4cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:57.263763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e371d-c767-476d-9e3d-9f62285b4cb5 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:57.264037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e371d-c767-476d-9e3d-9f62285b4cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:57.264303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e371d-c767-476d-9e3d-9f62285b4cb5 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:57.264393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4917321-5d2e-4993-90eb-9755ed777bc6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:57.264968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e371d-c767-476d-9e3d-9f62285b4cb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:57.266441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e371d-c767-476d-9e3d-9f62285b4cb5 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:57.299090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bc984-e083-4467-b979-f85296e57d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:57.299694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bc984-e083-4467-b979-f85296e57d0c None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:57.299967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bc984-e083-4467-b979-f85296e57d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:57.300235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bc984-e083-4467-b979-f85296e57d0c None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:57.300900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bc984-e083-4467-b979-f85296e57d0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:57.302325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bc984-e083-4467-b979-f85296e57d0c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:57.432203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 236/469] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:57 2022] POST /identity/v3/users => generated 246 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:57.433587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec9f49-bec9-44e9-81ac-02309679b9dd None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:57.433868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec9f49-bec9-44e9-81ac-02309679b9dd None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:57.434135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec9f49-bec9-44e9-81ac-02309679b9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:57.443904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 234/470] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:57 2022] POST /identity/v3/users => generated 248 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:57.456996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bc984-e083-4467-b979-f85296e57d0c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:57.471435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-bdec9f49-bec9-44e9-81ac-02309679b9dd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:57.472824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-bdec9f49-bec9-44e9-81ac-02309679b9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:57.576624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 237/471] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:57.578139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221ad68-6fca-48d0-8087-c3f8707b30d6 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:57.578423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221ad68-6fca-48d0-8087-c3f8707b30d6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:57.578687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221ad68-6fca-48d0-8087-c3f8707b30d6 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:57.584888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 238/472] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:57.586433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71189b3-ff2e-446c-b17b-53babc99dbde None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:57.586951 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:57.588173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71189b3-ff2e-446c-b17b-53babc99dbde None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:57.588613 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71189b3-ff2e-446c-b17b-53babc99dbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:57.593593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bc984-e083-4467-b979-f85296e57d0c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:57.610285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a71189b3-ff2e-446c-b17b-53babc99dbde None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:57.611716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a71189b3-ff2e-446c-b17b-53babc99dbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:57.633709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459187d-ffb5-4fed-8b75-8850762004dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:57.634616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459187d-ffb5-4fed-8b75-8850762004dc None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:57.634907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459187d-ffb5-4fed-8b75-8850762004dc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:57.635174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459187d-ffb5-4fed-8b75-8850762004dc None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:57.635799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459187d-ffb5-4fed-8b75-8850762004dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:57.637290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459187d-ffb5-4fed-8b75-8850762004dc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:57.695667 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 239/473] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:11:57.742599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 235/474] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:57 2022] POST /identity/v3/users => generated 248 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:57.891405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71189b3-ff2e-446c-b17b-53babc99dbde None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:57.908376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:57.998100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459187d-ffb5-4fed-8b75-8850762004dc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:58.001111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71189b3-ff2e-446c-b17b-53babc99dbde None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:58.057863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761230-6c3c-4cd2-8cbb-71b5e4a0b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:58.059687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761230-6c3c-4cd2-8cbb-71b5e4a0b0ce None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:58.060267 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761230-6c3c-4cd2-8cbb-71b5e4a0b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:58.060682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761230-6c3c-4cd2-8cbb-71b5e4a0b0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:58.076072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761230-6c3c-4cd2-8cbb-71b5e4a0b0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:58.078690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761230-6c3c-4cd2-8cbb-71b5e4a0b0ce None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-36761230-6c3c-4cd2-8cbb-71b5e4a0b0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:11:58.098784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:58.099912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:11:58.100958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:11:58.100958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:11:58.100958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:11:58.103390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 240/475] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:58.109143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761230-6c3c-4cd2-8cbb-71b5e4a0b0ce None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:58.130831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459187d-ffb5-4fed-8b75-8850762004dc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:58.178400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:58.179662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:58.181161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:58.181720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:58.183032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:58.186230 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:58.193661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:58.226395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.sql.core [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 11:11:58.226395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 11:11:58.226395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [parameters: {'id': '02456fe952814b12a9d07014698f9b78', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 11:11:58.226395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81706) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 11:11:58.240519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761230-6c3c-4cd2-8cbb-71b5e4a0b0ce None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result.read() Aug 24 11:11:58.256101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:11:58.257013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application yield resource Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise self.value Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application session.commit() Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 11:11:58.258333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.session.flush() Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application compat.raise_( Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 11:11:58.260175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:11:58.261444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result.read() Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:11:58.263754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [parameters: {'id': '02456fe952814b12a9d07014698f9b78', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:58.266048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 11:11:58.268118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:58.271547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 236/476] 10.222.0.84 () {60 vars in 1131 bytes} [Wed Aug 24 11:11:57 2022] POST /identity/v3/roles => generated 144 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:58.271547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:58.299761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c339b34-3ec5-49e1-8e0d-ad660105054a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:58.301293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c339b34-3ec5-49e1-8e0d-ad660105054a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:58.302993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c339b34-3ec5-49e1-8e0d-ad660105054a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:58.304368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c339b34-3ec5-49e1-8e0d-ad660105054a None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:58.305636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c339b34-3ec5-49e1-8e0d-ad660105054a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:58.307203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c339b34-3ec5-49e1-8e0d-ad660105054a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:58.310043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c339b34-3ec5-49e1-8e0d-ad660105054a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:58.316033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:58.330323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 241/477] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:11:58.338649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c339b34-3ec5-49e1-8e0d-ad660105054a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:58.409816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32749cd1-4bef-473b-a918-e279fcc9b4a2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:58.447540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa953f5-8322-443c-b8ff-53df69c0144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:58.448585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa953f5-8322-443c-b8ff-53df69c0144b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:58.449245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa953f5-8322-443c-b8ff-53df69c0144b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:58.449801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa953f5-8322-443c-b8ff-53df69c0144b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:58.461063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa953f5-8322-443c-b8ff-53df69c0144b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-8aa953f5-8322-443c-b8ff-53df69c0144b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:11:58.471449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:11:58.472378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 11:11:58.473210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:11:58.478325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 237/478] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:11:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:11:58.482572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa953f5-8322-443c-b8ff-53df69c0144b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:11:58.528657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c339b34-3ec5-49e1-8e0d-ad660105054a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:58.589174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:58.613407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e4091-70d5-4dc2-869c-9259dcc90dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:58.614393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e4091-70d5-4dc2-869c-9259dcc90dcf None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:58.614871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e4091-70d5-4dc2-869c-9259dcc90dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:58.614973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:11:58.615467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e4091-70d5-4dc2-869c-9259dcc90dcf None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:58.616325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e4091-70d5-4dc2-869c-9259dcc90dcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:58.618088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e4091-70d5-4dc2-869c-9259dcc90dcf None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:58.640621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa953f5-8322-443c-b8ff-53df69c0144b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:11:58.683597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb48fa9-081e-45d2-baa8-ea6cf8fd3c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:11:58.684214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48fa9-081e-45d2-baa8-ea6cf8fd3c32 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:58.684492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48fa9-081e-45d2-baa8-ea6cf8fd3c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:58.684765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48fa9-081e-45d2-baa8-ea6cf8fd3c32 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:58.685461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb48fa9-081e-45d2-baa8-ea6cf8fd3c32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:11:58.685815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb48fa9-081e-45d2-baa8-ea6cf8fd3c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:11:58.687466 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb48fa9-081e-45d2-baa8-ea6cf8fd3c32 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:11:58.696197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 238/479] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:11:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:11:58.778914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 242/480] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:11:58 2022] POST /identity/v3/users => generated 249 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:11:59.660821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819c1a-95ba-42f3-adce-28f74f3b099e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:59.660821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819c1a-95ba-42f3-adce-28f74f3b099e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:59.660821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c819c1a-95ba-42f3-adce-28f74f3b099e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:59.660821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 239/481] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:11:59.666422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55862dc-5ded-4c03-80de-75f655c35ca9 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:11:59.666820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55862dc-5ded-4c03-80de-75f655c35ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:11:59.667002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55862dc-5ded-4c03-80de-75f655c35ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:11:59.694073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d55862dc-5ded-4c03-80de-75f655c35ca9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:11:59.695263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d55862dc-5ded-4c03-80de-75f655c35ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:11:59.791383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 243/482] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:00.187431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c203eb8-2860-4ea7-b968-8d3d0ce2b5f5 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:00.188065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c203eb8-2860-4ea7-b968-8d3d0ce2b5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:00.188585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c203eb8-2860-4ea7-b968-8d3d0ce2b5f5 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:00.190393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 240/483] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:00.193129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55862dc-5ded-4c03-80de-75f655c35ca9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:00.195330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7f627-80c8-471b-af5f-b69ddf163dbd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:00.195817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7f627-80c8-471b-af5f-b69ddf163dbd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:00.197009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7f627-80c8-471b-af5f-b69ddf163dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:00.236341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-22e7f627-80c8-471b-af5f-b69ddf163dbd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:00.241752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-22e7f627-80c8-471b-af5f-b69ddf163dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:00.368693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 241/484] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:00.370173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef9d4-4895-47bb-9189-29aa8e3960a0 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:00.370455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef9d4-4895-47bb-9189-29aa8e3960a0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:00.370725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ef9d4-4895-47bb-9189-29aa8e3960a0 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:00.371783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55862dc-5ded-4c03-80de-75f655c35ca9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:00.372237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 242/485] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:00.378298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d3c4f-ea69-4988-ab93-52136994a7dd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:00.378578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d3c4f-ea69-4988-ab93-52136994a7dd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:00.378851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d3c4f-ea69-4988-ab93-52136994a7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:00.388114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:00.401142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-223d3c4f-ea69-4988-ab93-52136994a7dd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:00.402504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-223d3c4f-ea69-4988-ab93-52136994a7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:00.421209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d201bcb-631a-43b1-afcb-b2a18c61d310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:00.421861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d201bcb-631a-43b1-afcb-b2a18c61d310 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:00.422139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d201bcb-631a-43b1-afcb-b2a18c61d310 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:00.422403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d201bcb-631a-43b1-afcb-b2a18c61d310 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:00.434238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d201bcb-631a-43b1-afcb-b2a18c61d310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8d201bcb-631a-43b1-afcb-b2a18c61d310 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:00.439124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:00.440015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:00.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 244/486] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:00.445787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d201bcb-631a-43b1-afcb-b2a18c61d310 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:00.525254 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:00.532224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 243/487] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:00.588208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d201bcb-631a-43b1-afcb-b2a18c61d310 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:00.623755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4074b-92a8-40ee-abe6-6733195ac5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:00.624363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4074b-92a8-40ee-abe6-6733195ac5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:00.624645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4074b-92a8-40ee-abe6-6733195ac5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:00.624945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4074b-92a8-40ee-abe6-6733195ac5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:00.625603 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4074b-92a8-40ee-abe6-6733195ac5a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:00.625902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4074b-92a8-40ee-abe6-6733195ac5a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:00.627266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4074b-92a8-40ee-abe6-6733195ac5a4 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:00.635958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 245/488] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:00.641194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d3c4f-ea69-4988-ab93-52136994a7dd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:00.779225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4074b-92a8-40ee-abe6-6733195ac5a4 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:00.802949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d3c4f-ea69-4988-ab93-52136994a7dd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:00.844823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:00.851137 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b431154-949a-40f6-9a47-165e3d4c5765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:00.851774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b431154-949a-40f6-9a47-165e3d4c5765 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:00.852050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b431154-949a-40f6-9a47-165e3d4c5765 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:00.852862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b431154-949a-40f6-9a47-165e3d4c5765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:00.872535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b431154-949a-40f6-9a47-165e3d4c5765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:00.874091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b431154-949a-40f6-9a47-165e3d4c5765 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-4b431154-949a-40f6-9a47-165e3d4c5765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:00.890431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:00.891622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:00.892434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:00.892434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:00.892434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:00.892434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 244/489] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:00.892434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b431154-949a-40f6-9a47-165e3d4c5765 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:00.935914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4074b-92a8-40ee-abe6-6733195ac5a4 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:00.960159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:00.982930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124a998-573d-41b9-8250-0b61ef2e9226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:00.984647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124a998-573d-41b9-8250-0b61ef2e9226 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:00.985120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124a998-573d-41b9-8250-0b61ef2e9226 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:00.985538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124a998-573d-41b9-8250-0b61ef2e9226 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:00.999771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124a998-573d-41b9-8250-0b61ef2e9226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-9124a998-573d-41b9-8250-0b61ef2e9226 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:01.006905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:01.007969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:01.009681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:01.012379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 246/490] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:01.013962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124a998-573d-41b9-8250-0b61ef2e9226 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.019380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b431154-949a-40f6-9a47-165e3d4c5765 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.063348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576937b8-b499-4da7-a71f-019742668bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.064248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576937b8-b499-4da7-a71f-019742668bec None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.064676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576937b8-b499-4da7-a71f-019742668bec None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.065570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576937b8-b499-4da7-a71f-019742668bec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.080680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576937b8-b499-4da7-a71f-019742668bec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-576937b8-b499-4da7-a71f-019742668bec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:01.089812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:01.090904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:01.091855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:01.094317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 245/491] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:01.095294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576937b8-b499-4da7-a71f-019742668bec None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.130956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:01.140480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124a998-573d-41b9-8250-0b61ef2e9226 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.176873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd66526-89cf-473b-8c93-5c04bd46d57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.177978 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd66526-89cf-473b-8c93-5c04bd46d57b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.177978 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd66526-89cf-473b-8c93-5c04bd46d57b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.177978 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd66526-89cf-473b-8c93-5c04bd46d57b None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.178593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd66526-89cf-473b-8c93-5c04bd46d57b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:01.178886 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd66526-89cf-473b-8c93-5c04bd46d57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:01.180238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd66526-89cf-473b-8c93-5c04bd46d57b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:01.193205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 247/492] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:01.198851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd66526-89cf-473b-8c93-5c04bd46d57b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.209206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576937b8-b499-4da7-a71f-019742668bec None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.220599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:01.245761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7f590-7bb7-47b6-bb80-44c7524a0f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.246363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7f590-7bb7-47b6-bb80-44c7524a0f54 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.246639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7f590-7bb7-47b6-bb80-44c7524a0f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.246966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7f590-7bb7-47b6-bb80-44c7524a0f54 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.247808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7f590-7bb7-47b6-bb80-44c7524a0f54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:01.248110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7f590-7bb7-47b6-bb80-44c7524a0f54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:01.249554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7f590-7bb7-47b6-bb80-44c7524a0f54 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:01.257901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 246/493] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:01.259804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2450c6c-d926-4ed5-89c3-70a2d954d411 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.260096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2450c6c-d926-4ed5-89c3-70a2d954d411 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.260370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2450c6c-d926-4ed5-89c3-70a2d954d411 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.261597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 247/494] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:01.264926 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2450c6c-d926-4ed5-89c3-70a2d954d411 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.305445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd66526-89cf-473b-8c93-5c04bd46d57b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.338953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29830861-93d2-43ab-ac45-abbb6c48ce95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.339481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29830861-93d2-43ab-ac45-abbb6c48ce95 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.339761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29830861-93d2-43ab-ac45-abbb6c48ce95 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.340032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29830861-93d2-43ab-ac45-abbb6c48ce95 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.340691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29830861-93d2-43ab-ac45-abbb6c48ce95 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:01.341018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29830861-93d2-43ab-ac45-abbb6c48ce95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:01.342391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29830861-93d2-43ab-ac45-abbb6c48ce95 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:01.350145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 248/495] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:01.351286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca27c7-e8e4-427b-a9d2-ec09a9cbe485 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.351567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca27c7-e8e4-427b-a9d2-ec09a9cbe485 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.351834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca27c7-e8e4-427b-a9d2-ec09a9cbe485 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.353723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 249/496] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:01.357888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca27c7-e8e4-427b-a9d2-ec09a9cbe485 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.365480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2450c6c-d926-4ed5-89c3-70a2d954d411 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.373733 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:01.399818 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150478a4-240d-4fb2-bacc-c5f3c64d0138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.400467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150478a4-240d-4fb2-bacc-c5f3c64d0138 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.400766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150478a4-240d-4fb2-bacc-c5f3c64d0138 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.401051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150478a4-240d-4fb2-bacc-c5f3c64d0138 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.410111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150478a4-240d-4fb2-bacc-c5f3c64d0138 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:01.411554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150478a4-240d-4fb2-bacc-c5f3c64d0138 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-150478a4-240d-4fb2-bacc-c5f3c64d0138 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:01.424416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:01.425444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:01.426311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 248/497] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:01.426881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca86c2-5109-41d7-b9e1-a9e442b2ebfa None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.426881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca86c2-5109-41d7-b9e1-a9e442b2ebfa None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.427160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca86c2-5109-41d7-b9e1-a9e442b2ebfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.445300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-38ca86c2-5109-41d7-b9e1-a9e442b2ebfa None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:01.446580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-38ca86c2-5109-41d7-b9e1-a9e442b2ebfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:01.463313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca27c7-e8e4-427b-a9d2-ec09a9cbe485 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.472201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:01.498363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36065fc9-f761-4c82-b989-a5e5ab12c5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.499013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36065fc9-f761-4c82-b989-a5e5ab12c5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.510334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36065fc9-f761-4c82-b989-a5e5ab12c5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.510334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36065fc9-f761-4c82-b989-a5e5ab12c5f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.510557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36065fc9-f761-4c82-b989-a5e5ab12c5f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:01.511826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36065fc9-f761-4c82-b989-a5e5ab12c5f9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:01.514741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 249/498] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:01.515568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca86c2-5109-41d7-b9e1-a9e442b2ebfa None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-36065fc9-f761-4c82-b989-a5e5ab12c5f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:01.535095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:01.536043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:01.536918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:01.536918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:01.536918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:01.536918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 250/499] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:01.537554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e473ba1-2a23-402b-9ca9-2801f19c4382 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.537656 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e473ba1-2a23-402b-9ca9-2801f19c4382 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.538190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e473ba1-2a23-402b-9ca9-2801f19c4382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.557395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-8e473ba1-2a23-402b-9ca9-2801f19c4382 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:01.558584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8e473ba1-2a23-402b-9ca9-2801f19c4382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:01.620482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:01.636823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 251/500] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:01.636823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e473ba1-2a23-402b-9ca9-2801f19c4382 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.641231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca86c2-5109-41d7-b9e1-a9e442b2ebfa None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.695786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c1416-6959-4b66-9d52-88a07e1f3daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.696825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c1416-6959-4b66-9d52-88a07e1f3daf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.697417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c1416-6959-4b66-9d52-88a07e1f3daf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.697786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c1416-6959-4b66-9d52-88a07e1f3daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.711102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c1416-6959-4b66-9d52-88a07e1f3daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:01.713714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3c1416-6959-4b66-9d52-88a07e1f3daf None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:01.733724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e473ba1-2a23-402b-9ca9-2801f19c4382 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-eb3c1416-6959-4b66-9d52-88a07e1f3daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:01.734667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:01.735683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:01.736553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:01.736553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:01.736553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:01.738249 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 250/501] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:01.739120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c1416-6959-4b66-9d52-88a07e1f3daf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.761909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:01.772995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48199f-ac13-43bb-a2b8-91a58ef2c444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.773874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48199f-ac13-43bb-a2b8-91a58ef2c444 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.774305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48199f-ac13-43bb-a2b8-91a58ef2c444 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.774647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48199f-ac13-43bb-a2b8-91a58ef2c444 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.775447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48199f-ac13-43bb-a2b8-91a58ef2c444 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:01.775819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48199f-ac13-43bb-a2b8-91a58ef2c444 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:01.777352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48199f-ac13-43bb-a2b8-91a58ef2c444 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:01.817137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 252/502] 10.222.0.84 () {56 vars in 1137 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:01.817137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48199f-ac13-43bb-a2b8-91a58ef2c444 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.863365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c1416-6959-4b66-9d52-88a07e1f3daf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.874524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:01.903558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18c086-e46c-49b1-a0ea-106f00ab794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.904151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18c086-e46c-49b1-a0ea-106f00ab794c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.904422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18c086-e46c-49b1-a0ea-106f00ab794c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.904689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18c086-e46c-49b1-a0ea-106f00ab794c None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.905511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18c086-e46c-49b1-a0ea-106f00ab794c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:01.905811 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18c086-e46c-49b1-a0ea-106f00ab794c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:01.907223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18c086-e46c-49b1-a0ea-106f00ab794c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:01.918355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48199f-ac13-43bb-a2b8-91a58ef2c444 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:01.924078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 251/503] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:01.924078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18c086-e46c-49b1-a0ea-106f00ab794c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.955139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5a6b0-5299-46c4-a685-8621b16fc2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:01.955765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5a6b0-5299-46c4-a685-8621b16fc2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:01.956047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5a6b0-5299-46c4-a685-8621b16fc2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:01.956318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5a6b0-5299-46c4-a685-8621b16fc2f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:01.956990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5a6b0-5299-46c4-a685-8621b16fc2f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:01.957289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5a6b0-5299-46c4-a685-8621b16fc2f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:01.958654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5a6b0-5299-46c4-a685-8621b16fc2f6 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:01.970012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 253/504] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:01.970791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5a6b0-5299-46c4-a685-8621b16fc2f6 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:01.998067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:02.022513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18c086-e46c-49b1-a0ea-106f00ab794c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.056635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b35da-fe6b-4689-ae45-f5bb182eceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.057290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b35da-fe6b-4689-ae45-f5bb182eceee None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.057568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b35da-fe6b-4689-ae45-f5bb182eceee None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.057839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b35da-fe6b-4689-ae45-f5bb182eceee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.067582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b35da-fe6b-4689-ae45-f5bb182eceee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.068555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5a6b0-5299-46c4-a685-8621b16fc2f6 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.069066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b35da-fe6b-4689-ae45-f5bb182eceee None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.082907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f55b35da-fe6b-4689-ae45-f5bb182eceee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:02.085143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:02.086131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:02.089961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:02.089961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:02.089961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:02.089961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 252/505] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:02.089961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b35da-fe6b-4689-ae45-f5bb182eceee None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.105548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71507c50-80e6-4049-b5bc-6d874c061389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.106154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71507c50-80e6-4049-b5bc-6d874c061389 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.106420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71507c50-80e6-4049-b5bc-6d874c061389 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.106686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71507c50-80e6-4049-b5bc-6d874c061389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.175874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71507c50-80e6-4049-b5bc-6d874c061389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.177305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71507c50-80e6-4049-b5bc-6d874c061389 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.187156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b35da-fe6b-4689-ae45-f5bb182eceee None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.212390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:02.220970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3a97-260b-4541-bbf0-4314cec39392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.221589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3a97-260b-4541-bbf0-4314cec39392 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.233089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3a97-260b-4541-bbf0-4314cec39392 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.233089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3a97-260b-4541-bbf0-4314cec39392 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.233089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e3a97-260b-4541-bbf0-4314cec39392 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.233089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e3a97-260b-4541-bbf0-4314cec39392 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-984e3a97-260b-4541-bbf0-4314cec39392 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:02.245806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:02.251654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 253/506] 10.222.0.84 () {56 vars in 1093 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:02.252843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3a97-260b-4541-bbf0-4314cec39392 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.287914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 254/507] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:02.288844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71507c50-80e6-4049-b5bc-6d874c061389 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.303466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:02.351269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3a97-260b-4541-bbf0-4314cec39392 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.385363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0087d49-ac31-4e7b-81d8-e66c7773a817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.385981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0087d49-ac31-4e7b-81d8-e66c7773a817 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.386253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0087d49-ac31-4e7b-81d8-e66c7773a817 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.386520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0087d49-ac31-4e7b-81d8-e66c7773a817 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.386956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71507c50-80e6-4049-b5bc-6d874c061389 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.398029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0087d49-ac31-4e7b-81d8-e66c7773a817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a0087d49-ac31-4e7b-81d8-e66c7773a817 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:02.403242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:02.404298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 254/508] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:02.405115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0087d49-ac31-4e7b-81d8-e66c7773a817 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.420785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db75a6-7ff3-4461-a8d3-dfb0b80062c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.421512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db75a6-7ff3-4461-a8d3-dfb0b80062c6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.421855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db75a6-7ff3-4461-a8d3-dfb0b80062c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.422195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db75a6-7ff3-4461-a8d3-dfb0b80062c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.429171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db75a6-7ff3-4461-a8d3-dfb0b80062c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.431123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db75a6-7ff3-4461-a8d3-dfb0b80062c6 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-b9db75a6-7ff3-4461-a8d3-dfb0b80062c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:02.443575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:02.444629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 255/509] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:02.445757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db75a6-7ff3-4461-a8d3-dfb0b80062c6 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.453716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:02.510812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0087d49-ac31-4e7b-81d8-e66c7773a817 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.524105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:02.544389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a507035-a8d2-4441-a559-233938dd80b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.545076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a507035-a8d2-4441-a559-233938dd80b1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.545351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a507035-a8d2-4441-a559-233938dd80b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.545625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a507035-a8d2-4441-a559-233938dd80b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.546256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a507035-a8d2-4441-a559-233938dd80b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:02.546553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a507035-a8d2-4441-a559-233938dd80b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.547946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a507035-a8d2-4441-a559-233938dd80b1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.548317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db75a6-7ff3-4461-a8d3-dfb0b80062c6 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.561411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 255/510] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:02.562208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a507035-a8d2-4441-a559-233938dd80b1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.584231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079187e-d423-4a80-a3f6-ff1ee02f72a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.584840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079187e-d423-4a80-a3f6-ff1ee02f72a2 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.585125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079187e-d423-4a80-a3f6-ff1ee02f72a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.585389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079187e-d423-4a80-a3f6-ff1ee02f72a2 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.586103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079187e-d423-4a80-a3f6-ff1ee02f72a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:02.586400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079187e-d423-4a80-a3f6-ff1ee02f72a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.588061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079187e-d423-4a80-a3f6-ff1ee02f72a2 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.605596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 256/511] 10.222.0.84 () {56 vars in 1133 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:02.606586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079187e-d423-4a80-a3f6-ff1ee02f72a2 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.668101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a507035-a8d2-4441-a559-233938dd80b1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.675393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:02.703703 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14686238-6ee2-4bd8-9f5a-008ae9a7b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.704251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686238-6ee2-4bd8-9f5a-008ae9a7b3fc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.704349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079187e-d423-4a80-a3f6-ff1ee02f72a2 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.704528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686238-6ee2-4bd8-9f5a-008ae9a7b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.704848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686238-6ee2-4bd8-9f5a-008ae9a7b3fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.715333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14686238-6ee2-4bd8-9f5a-008ae9a7b3fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.716768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14686238-6ee2-4bd8-9f5a-008ae9a7b3fc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-14686238-6ee2-4bd8-9f5a-008ae9a7b3fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:02.732241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:02.733337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:02.734365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:02.734365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:02.734365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:02.734365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 256/512] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:02.734644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14686238-6ee2-4bd8-9f5a-008ae9a7b3fc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.735793 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:02.741010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e453c7-0bad-401b-a586-093f5ec25ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.741606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e453c7-0bad-401b-a586-093f5ec25ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.741879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e453c7-0bad-401b-a586-093f5ec25ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.742144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e453c7-0bad-401b-a586-093f5ec25ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.742801 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e453c7-0bad-401b-a586-093f5ec25ae8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:02.743684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e453c7-0bad-401b-a586-093f5ec25ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.745245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e453c7-0bad-401b-a586-093f5ec25ae8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.753967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 257/513] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:02.754759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e453c7-0bad-401b-a586-093f5ec25ae8 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.852741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14686238-6ee2-4bd8-9f5a-008ae9a7b3fc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.857501 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e453c7-0bad-401b-a586-093f5ec25ae8 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:02.886748 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca935e6-02cf-48a2-8fd9-476d275b5a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.887339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca935e6-02cf-48a2-8fd9-476d275b5a5d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.887610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca935e6-02cf-48a2-8fd9-476d275b5a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.887875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca935e6-02cf-48a2-8fd9-476d275b5a5d None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.888582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca935e6-02cf-48a2-8fd9-476d275b5a5d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:02.888900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca935e6-02cf-48a2-8fd9-476d275b5a5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.890314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca935e6-02cf-48a2-8fd9-476d275b5a5d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.890746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8d328-58f1-4bff-8bac-5bbb7025b5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:02.891382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8d328-58f1-4bff-8bac-5bbb7025b5ce None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:02.891660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8d328-58f1-4bff-8bac-5bbb7025b5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:02.891932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8d328-58f1-4bff-8bac-5bbb7025b5ce None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:02.892558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8d328-58f1-4bff-8bac-5bbb7025b5ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:02.894006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8d328-58f1-4bff-8bac-5bbb7025b5ce None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:02.926379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 257/514] 10.222.0.84 () {56 vars in 1137 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:02.927292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca935e6-02cf-48a2-8fd9-476d275b5a5d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:02.945044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:02.981396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 258/515] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:12:02 2022] POST /identity/v3/users => generated 247 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:02.982543 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8d328-58f1-4bff-8bac-5bbb7025b5ce None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:03.033457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca935e6-02cf-48a2-8fd9-476d275b5a5d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:03.050407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:03.075663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a902c6-f046-43c1-bff1-8e282de2bb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:03.076304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a902c6-f046-43c1-bff1-8e282de2bb82 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:03.076618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a902c6-f046-43c1-bff1-8e282de2bb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:03.076957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a902c6-f046-43c1-bff1-8e282de2bb82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:03.108263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8d328-58f1-4bff-8bac-5bbb7025b5ce None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:03.153971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba3671-cf0c-46d9-a627-3370a928f052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:03.154664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba3671-cf0c-46d9-a627-3370a928f052 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:03.154957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba3671-cf0c-46d9-a627-3370a928f052 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:03.155211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba3671-cf0c-46d9-a627-3370a928f052 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:03.155847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba3671-cf0c-46d9-a627-3370a928f052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:03.156237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba3671-cf0c-46d9-a627-3370a928f052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:03.157557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba3671-cf0c-46d9-a627-3370a928f052 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:03.170974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a902c6-f046-43c1-bff1-8e282de2bb82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:03.171726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 259/516] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:03.172563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba3671-cf0c-46d9-a627-3370a928f052 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:03.172671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a902c6-f046-43c1-bff1-8e282de2bb82 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:03.191620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:03.282830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 258/517] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:03.282830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a902c6-f046-43c1-bff1-8e282de2bb82 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:03.288351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba3671-cf0c-46d9-a627-3370a928f052 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:03.300057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:03.324282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc10c2-4174-4dba-ad59-71ffa7e87ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:03.328872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc10c2-4174-4dba-ad59-71ffa7e87ddc None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:03.328872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc10c2-4174-4dba-ad59-71ffa7e87ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:03.328872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc10c2-4174-4dba-ad59-71ffa7e87ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:03.341508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc10c2-4174-4dba-ad59-71ffa7e87ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:03.342971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc10c2-4174-4dba-ad59-71ffa7e87ddc None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-2fbc10c2-4174-4dba-ad59-71ffa7e87ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:03.358508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:03.359585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:03.360538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:03.360538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:03.360538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:03.360538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 260/518] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:03.362565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc10c2-4174-4dba-ad59-71ffa7e87ddc None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:03.394139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a902c6-f046-43c1-bff1-8e282de2bb82 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:03.412733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:03.431336 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919a55d-78d8-4301-bef0-30e7aceb750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:03.431934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919a55d-78d8-4301-bef0-30e7aceb750e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:03.432212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919a55d-78d8-4301-bef0-30e7aceb750e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:03.432480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919a55d-78d8-4301-bef0-30e7aceb750e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:03.474931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc10c2-4174-4dba-ad59-71ffa7e87ddc None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:03.506809 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919a55d-78d8-4301-bef0-30e7aceb750e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:03.508267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919a55d-78d8-4301-bef0-30e7aceb750e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:03.512565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654be6c-ee79-4243-ae21-b823bc0427cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:03.513502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654be6c-ee79-4243-ae21-b823bc0427cf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:03.513916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654be6c-ee79-4243-ae21-b823bc0427cf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:03.514320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654be6c-ee79-4243-ae21-b823bc0427cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:03.522989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654be6c-ee79-4243-ae21-b823bc0427cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:03.525892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654be6c-ee79-4243-ae21-b823bc0427cf None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-b654be6c-ee79-4243-ae21-b823bc0427cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:03.541693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:03.542690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:03.543716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:03.543716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:03.543716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:03.543716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:03.543716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:03.543716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:03.543716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:03.543716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:03.546329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 261/519] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:03.547249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654be6c-ee79-4243-ae21-b823bc0427cf None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:03.574669 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:03.615854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 259/520] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:03.616965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919a55d-78d8-4301-bef0-30e7aceb750e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:03.636849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:03.662967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654be6c-ee79-4243-ae21-b823bc0427cf None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:03.699372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a7e41-02dd-41bd-8823-77c80bb524e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:03.700006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a7e41-02dd-41bd-8823-77c80bb524e5 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:03.700282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a7e41-02dd-41bd-8823-77c80bb524e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:03.700551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a7e41-02dd-41bd-8823-77c80bb524e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:03.711229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a7e41-02dd-41bd-8823-77c80bb524e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:03.712952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a7e41-02dd-41bd-8823-77c80bb524e5 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-150a7e41-02dd-41bd-8823-77c80bb524e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:03.727792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:03.728671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:03.729514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:03.729514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:03.729514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:03.732296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919a55d-78d8-4301-bef0-30e7aceb750e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:03.733934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 262/521] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:03.734775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a7e41-02dd-41bd-8823-77c80bb524e5 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:03.768528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06e53c-5255-46b3-8eee-37fccccc0614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:03.769154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06e53c-5255-46b3-8eee-37fccccc0614 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:03.769429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06e53c-5255-46b3-8eee-37fccccc0614 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:03.769701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06e53c-5255-46b3-8eee-37fccccc0614 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:03.770326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06e53c-5255-46b3-8eee-37fccccc0614 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:03.770622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06e53c-5255-46b3-8eee-37fccccc0614 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:03.771980 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06e53c-5255-46b3-8eee-37fccccc0614 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:03.783950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 260/522] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:03.784575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06e53c-5255-46b3-8eee-37fccccc0614 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:03.816366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:03.842115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a7e41-02dd-41bd-8823-77c80bb524e5 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:03.867566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:03.879325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b465e-9c6a-4f1c-925b-28c5a49b7360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:03.879924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b465e-9c6a-4f1c-925b-28c5a49b7360 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:03.880245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b465e-9c6a-4f1c-925b-28c5a49b7360 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:03.880519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b465e-9c6a-4f1c-925b-28c5a49b7360 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:03.881235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b465e-9c6a-4f1c-925b-28c5a49b7360 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:03.881573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b465e-9c6a-4f1c-925b-28c5a49b7360 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:03.883006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b465e-9c6a-4f1c-925b-28c5a49b7360 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:03.891056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06e53c-5255-46b3-8eee-37fccccc0614 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:03.902843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 263/523] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:03.904195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b465e-9c6a-4f1c-925b-28c5a49b7360 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:03.926838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ce4fb-5493-4f25-a8ae-43e28582831d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:03.927487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ce4fb-5493-4f25-a8ae-43e28582831d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:03.927764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ce4fb-5493-4f25-a8ae-43e28582831d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:03.928032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ce4fb-5493-4f25-a8ae-43e28582831d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:03.939062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ce4fb-5493-4f25-a8ae-43e28582831d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:03.940514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ce4fb-5493-4f25-a8ae-43e28582831d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-2d2ce4fb-5493-4f25-a8ae-43e28582831d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:03.954622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:03.955532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:03.956462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:03.956462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:03.956462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:03.956462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 261/524] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:03.956671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ce4fb-5493-4f25-a8ae-43e28582831d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.016865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b465e-9c6a-4f1c-925b-28c5a49b7360 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.053798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab7953-6286-484b-90a2-95630b58d6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.054410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab7953-6286-484b-90a2-95630b58d6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.054683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab7953-6286-484b-90a2-95630b58d6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.054950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab7953-6286-484b-90a2-95630b58d6d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.056122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab7953-6286-484b-90a2-95630b58d6d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:04.056414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab7953-6286-484b-90a2-95630b58d6d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:04.057825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab7953-6286-484b-90a2-95630b58d6d3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:04.059321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:04.083305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 264/525] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:04.083305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab7953-6286-484b-90a2-95630b58d6d3 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.083557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ce4fb-5493-4f25-a8ae-43e28582831d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.109521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:04.119590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c5867-1fcf-45b5-8ba9-55708204181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.120367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c5867-1fcf-45b5-8ba9-55708204181a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.121036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c5867-1fcf-45b5-8ba9-55708204181a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.121036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c5867-1fcf-45b5-8ba9-55708204181a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.132048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c5867-1fcf-45b5-8ba9-55708204181a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:04.134730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c5867-1fcf-45b5-8ba9-55708204181a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-848c5867-1fcf-45b5-8ba9-55708204181a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:04.146893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:04.147815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 262/526] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:04.149568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c5867-1fcf-45b5-8ba9-55708204181a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.179932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab7953-6286-484b-90a2-95630b58d6d3 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.221619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a40d23-877b-4171-9383-e7bf973691b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.222482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a40d23-877b-4171-9383-e7bf973691b3 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.222931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a40d23-877b-4171-9383-e7bf973691b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.223199 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a40d23-877b-4171-9383-e7bf973691b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.223926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a40d23-877b-4171-9383-e7bf973691b3 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=75ac781cf2d84d95a789724e74e12acf) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:04.224230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a40d23-877b-4171-9383-e7bf973691b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:04.225711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a40d23-877b-4171-9383-e7bf973691b3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:04.231678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 265/527] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:04 2022] GET /identity/v3/role_assignments?user.id=75ac781cf2d84d95a789724e74e12acf&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:04.246655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a40d23-877b-4171-9383-e7bf973691b3 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.263854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c5867-1fcf-45b5-8ba9-55708204181a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.305302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ead2b-4ecf-401f-932d-36aa9e681b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.307219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ead2b-4ecf-401f-932d-36aa9e681b16 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.307503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ead2b-4ecf-401f-932d-36aa9e681b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.307773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ead2b-4ecf-401f-932d-36aa9e681b16 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.308438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ead2b-4ecf-401f-932d-36aa9e681b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:04.308738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ead2b-4ecf-401f-932d-36aa9e681b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:04.310176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ead2b-4ecf-401f-932d-36aa9e681b16 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:04.325331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 263/528] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:04.326330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ead2b-4ecf-401f-932d-36aa9e681b16 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.344969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:04.388295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:04.409086 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a40d23-877b-4171-9383-e7bf973691b3 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.441508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ead2b-4ecf-401f-932d-36aa9e681b16 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.457131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95813022-34e1-4217-9e4e-a9318cee4e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.458117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95813022-34e1-4217-9e4e-a9318cee4e9e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.458467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95813022-34e1-4217-9e4e-a9318cee4e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.458881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95813022-34e1-4217-9e4e-a9318cee4e9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.468389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95813022-34e1-4217-9e4e-a9318cee4e9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:04.470219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95813022-34e1-4217-9e4e-a9318cee4e9e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:04.479920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeaf484-3555-4186-9fac-ad50e0a9cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.480745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaf484-3555-4186-9fac-ad50e0a9cb85 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.481226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaf484-3555-4186-9fac-ad50e0a9cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.487219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaf484-3555-4186-9fac-ad50e0a9cb85 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.487219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeaf484-3555-4186-9fac-ad50e0a9cb85 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:04.487219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeaf484-3555-4186-9fac-ad50e0a9cb85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:04.487219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeaf484-3555-4186-9fac-ad50e0a9cb85 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-95813022-34e1-4217-9e4e-a9318cee4e9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:04.487482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:04.488257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 266/529] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:04.489609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95813022-34e1-4217-9e4e-a9318cee4e9e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.510317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 264/530] 10.222.0.84 () {56 vars in 1139 bytes} [Wed Aug 24 11:12:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:04.510317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeaf484-3555-4186-9fac-ad50e0a9cb85 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.614791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeaf484-3555-4186-9fac-ad50e0a9cb85 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.617806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95813022-34e1-4217-9e4e-a9318cee4e9e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.623765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:04.638895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:04.659628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e04337-2168-418b-894d-8724899bf76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.660459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e04337-2168-418b-894d-8724899bf76f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.660957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e04337-2168-418b-894d-8724899bf76f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.661372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e04337-2168-418b-894d-8724899bf76f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.663770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be25b62-7077-432a-b039-0f72c0b2a19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.664519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be25b62-7077-432a-b039-0f72c0b2a19c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.664638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be25b62-7077-432a-b039-0f72c0b2a19c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.665529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be25b62-7077-432a-b039-0f72c0b2a19c None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.666300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be25b62-7077-432a-b039-0f72c0b2a19c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:04.666603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be25b62-7077-432a-b039-0f72c0b2a19c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:04.668031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be25b62-7077-432a-b039-0f72c0b2a19c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:04.670995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e04337-2168-418b-894d-8724899bf76f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:04.672612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e04337-2168-418b-894d-8724899bf76f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-d6e04337-2168-418b-894d-8724899bf76f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:04.686569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:04.693478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:04.694310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:04.694310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:04.694310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:04.694310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:04.694310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:04.694310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:04.694310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:04.694310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:04.694711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 265/531] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:04.694711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be25b62-7077-432a-b039-0f72c0b2a19c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.695516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 267/532] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:04.697533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e04337-2168-418b-894d-8724899bf76f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.815186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be25b62-7077-432a-b039-0f72c0b2a19c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.819774 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e04337-2168-418b-894d-8724899bf76f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:04.860241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f51a1-03f0-400a-817e-79681c7713db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.861567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f51a1-03f0-400a-817e-79681c7713db None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.862386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f51a1-03f0-400a-817e-79681c7713db None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.863131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f51a1-03f0-400a-817e-79681c7713db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.868429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26629097-065a-4d11-9490-ac4e080b6da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:04.869017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26629097-065a-4d11-9490-ac4e080b6da8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:04.869299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26629097-065a-4d11-9490-ac4e080b6da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:04.869570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26629097-065a-4d11-9490-ac4e080b6da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:04.870299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26629097-065a-4d11-9490-ac4e080b6da8 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=f58a559951c94d68951da78c82341a00) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:04.870624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26629097-065a-4d11-9490-ac4e080b6da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:04.872038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26629097-065a-4d11-9490-ac4e080b6da8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:04.883133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 268/533] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:04 2022] GET /identity/v3/role_assignments?user.id=f58a559951c94d68951da78c82341a00&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:04.883133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26629097-065a-4d11-9490-ac4e080b6da8 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:04.940604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:04.981540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:05.018666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f51a1-03f0-400a-817e-79681c7713db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:05.020917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f51a1-03f0-400a-817e-79681c7713db None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:05.065802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26629097-065a-4d11-9490-ac4e080b6da8 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:05.103602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78982d-e54f-49fd-a89f-3d6e50e00024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:05.104314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78982d-e54f-49fd-a89f-3d6e50e00024 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:05.104594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78982d-e54f-49fd-a89f-3d6e50e00024 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:05.104963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78982d-e54f-49fd-a89f-3d6e50e00024 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:05.105695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78982d-e54f-49fd-a89f-3d6e50e00024 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:05.105995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78982d-e54f-49fd-a89f-3d6e50e00024 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:05.107694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78982d-e54f-49fd-a89f-3d6e50e00024 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:05.127217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 269/534] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:05.128653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b60041-6d02-4ff9-b0a6-523da9df8df9 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:05.129012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b60041-6d02-4ff9-b0a6-523da9df8df9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:05.129288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b60041-6d02-4ff9-b0a6-523da9df8df9 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:05.130775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 270/535] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:05.131600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b60041-6d02-4ff9-b0a6-523da9df8df9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:05.151919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 266/536] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:05.153294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2268e-8b9b-4756-baab-8442a14af519 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:05.153577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2268e-8b9b-4756-baab-8442a14af519 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:05.153846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2268e-8b9b-4756-baab-8442a14af519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:05.176822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1bd2268e-8b9b-4756-baab-8442a14af519 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:05.181179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1bd2268e-8b9b-4756-baab-8442a14af519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:05.254326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b60041-6d02-4ff9-b0a6-523da9df8df9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:05.256083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:05.260849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:05.269701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 267/537] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:05.270535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2268e-8b9b-4756-baab-8442a14af519 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:05.314762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a37d55-8f93-4325-a923-de36481d2197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:05.316173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a37d55-8f93-4325-a923-de36481d2197 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:05.316807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a37d55-8f93-4325-a923-de36481d2197 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:05.317289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a37d55-8f93-4325-a923-de36481d2197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:05.318018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a37d55-8f93-4325-a923-de36481d2197 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=d5ce16d5ea8240debfcb69ac7b1f71c0) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:05.318320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a37d55-8f93-4325-a923-de36481d2197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:05.323019 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a37d55-8f93-4325-a923-de36481d2197 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:05.333537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 271/538] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:05 2022] GET /identity/v3/role_assignments?user.id=d5ce16d5ea8240debfcb69ac7b1f71c0&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:05.409029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2268e-8b9b-4756-baab-8442a14af519 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:05.454847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac58a68-2ac1-404a-822b-af192e2c3470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:05.455491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac58a68-2ac1-404a-822b-af192e2c3470 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:05.455768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac58a68-2ac1-404a-822b-af192e2c3470 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:05.456041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac58a68-2ac1-404a-822b-af192e2c3470 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:05.467261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac58a68-2ac1-404a-822b-af192e2c3470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:05.468719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac58a68-2ac1-404a-822b-af192e2c3470 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-aac58a68-2ac1-404a-822b-af192e2c3470 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:05.485166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:05.486128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:05.487603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:05.487603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:05.487603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:05.487603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 268/539] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:05.491931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a37d55-8f93-4325-a923-de36481d2197 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:05.614788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac58a68-2ac1-404a-822b-af192e2c3470 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:05.655790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a37d55-8f93-4325-a923-de36481d2197 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:05.693573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:05.710050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46808e5f-b8e3-416b-97d3-9fe9cb5a0bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:05.710959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46808e5f-b8e3-416b-97d3-9fe9cb5a0bcd None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:05.711470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46808e5f-b8e3-416b-97d3-9fe9cb5a0bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:05.712242 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46808e5f-b8e3-416b-97d3-9fe9cb5a0bcd None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:05.713272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46808e5f-b8e3-416b-97d3-9fe9cb5a0bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:05.713791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46808e5f-b8e3-416b-97d3-9fe9cb5a0bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:05.715345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46808e5f-b8e3-416b-97d3-9fe9cb5a0bcd None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:05.728297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 272/540] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:05.734213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46808e5f-b8e3-416b-97d3-9fe9cb5a0bcd None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:05.764506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:05.827115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac58a68-2ac1-404a-822b-af192e2c3470 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:05.843360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46808e5f-b8e3-416b-97d3-9fe9cb5a0bcd None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:05.879338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb6716-d97c-416c-95d3-e5d0cd1a4f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:05.880304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb6716-d97c-416c-95d3-e5d0cd1a4f2e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:05.880854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb6716-d97c-416c-95d3-e5d0cd1a4f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:05.881322 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb6716-d97c-416c-95d3-e5d0cd1a4f2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:05.889692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb6716-d97c-416c-95d3-e5d0cd1a4f2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:05.891875 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb6716-d97c-416c-95d3-e5d0cd1a4f2e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:05.902701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d150f6a-f293-43f2-b805-c9823dd69544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:05.903343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d150f6a-f293-43f2-b805-c9823dd69544 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:05.903621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d150f6a-f293-43f2-b805-c9823dd69544 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:05.903896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d150f6a-f293-43f2-b805-c9823dd69544 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-90bb6716-d97c-416c-95d3-e5d0cd1a4f2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:05.906483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:05.907332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:05.908138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:05.908138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:05.908138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:05.908138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:05.908138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:05.908138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:05.908138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:05.908138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:05.910657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 273/541] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:05.914835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d150f6a-f293-43f2-b805-c9823dd69544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:05.916986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb6716-d97c-416c-95d3-e5d0cd1a4f2e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3d150f6a-f293-43f2-b805-c9823dd69544 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:05.920761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:05.921665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:05.922450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 269/542] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:05.941012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d150f6a-f293-43f2-b805-c9823dd69544 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:06.017463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:06.034578 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb6716-d97c-416c-95d3-e5d0cd1a4f2e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:06.064122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d150f6a-f293-43f2-b805-c9823dd69544 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:06.077194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:06.108717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874cd01-3f8e-41a8-b208-457934cd0195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:06.109364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874cd01-3f8e-41a8-b208-457934cd0195 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:06.111445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874cd01-3f8e-41a8-b208-457934cd0195 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:06.111731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874cd01-3f8e-41a8-b208-457934cd0195 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:06.112409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874cd01-3f8e-41a8-b208-457934cd0195 None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:06.112716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874cd01-3f8e-41a8-b208-457934cd0195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:06.114121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874cd01-3f8e-41a8-b208-457934cd0195 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:06.124281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3a73c-82a9-47cb-99a9-f5ea5146a1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:06.124916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a73c-82a9-47cb-99a9-f5ea5146a1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:06.125200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a73c-82a9-47cb-99a9-f5ea5146a1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:06.125474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3a73c-82a9-47cb-99a9-f5ea5146a1b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:06.126173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3a73c-82a9-47cb-99a9-f5ea5146a1b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:06.126481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3a73c-82a9-47cb-99a9-f5ea5146a1b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:06.127926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3a73c-82a9-47cb-99a9-f5ea5146a1b4 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:06.133777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 270/543] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:06.139147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874cd01-3f8e-41a8-b208-457934cd0195 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:06.156400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 274/544] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:06.161736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3a73c-82a9-47cb-99a9-f5ea5146a1b4 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:06.302446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3a73c-82a9-47cb-99a9-f5ea5146a1b4 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:06.351652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47a2eb-d5cd-4ebc-bd1f-bb52ac37a3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:06.352814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47a2eb-d5cd-4ebc-bd1f-bb52ac37a3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:06.353103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47a2eb-d5cd-4ebc-bd1f-bb52ac37a3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:06.353618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47a2eb-d5cd-4ebc-bd1f-bb52ac37a3e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:06.354346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47a2eb-d5cd-4ebc-bd1f-bb52ac37a3e9 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=372e96bd3361434e9169b783f0c9ea13) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:06.359053 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47a2eb-d5cd-4ebc-bd1f-bb52ac37a3e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:06.360633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47a2eb-d5cd-4ebc-bd1f-bb52ac37a3e9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:06.371728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 275/545] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:06 2022] GET /identity/v3/role_assignments?user.id=372e96bd3361434e9169b783f0c9ea13&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:06.397410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874cd01-3f8e-41a8-b208-457934cd0195 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:06.435883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d896e-61ef-4b21-b197-50f70749d1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:06.436587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d896e-61ef-4b21-b197-50f70749d1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:06.436895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d896e-61ef-4b21-b197-50f70749d1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:06.437166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d896e-61ef-4b21-b197-50f70749d1f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:06.449279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d896e-61ef-4b21-b197-50f70749d1f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:06.450864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d896e-61ef-4b21-b197-50f70749d1f5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-7f7d896e-61ef-4b21-b197-50f70749d1f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:06.467486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:06.468436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:06.469274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:06.469274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:06.469274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:06.469274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 271/546] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:06.470647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63246d73-309a-4d98-9646-2cbb6613a8cb None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:06.470938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63246d73-309a-4d98-9646-2cbb6613a8cb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:06.471207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63246d73-309a-4d98-9646-2cbb6613a8cb None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:06.472746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 276/547] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:12:06.478614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d896e-61ef-4b21-b197-50f70749d1f5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:06.478939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef663a1-b13f-4a64-87e1-eb1239a59fd2 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:06.479226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef663a1-b13f-4a64-87e1-eb1239a59fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:06.479500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef663a1-b13f-4a64-87e1-eb1239a59fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:06.498327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-eef663a1-b13f-4a64-87e1-eb1239a59fd2 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:06.499777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-eef663a1-b13f-4a64-87e1-eb1239a59fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:06.504463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:06.529174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:06.665174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 277/548] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:06.667720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d896e-61ef-4b21-b197-50f70749d1f5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:06.739289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b807310-dba8-4f60-a430-6f6d71296532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:06.739919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b807310-dba8-4f60-a430-6f6d71296532 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:06.740233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b807310-dba8-4f60-a430-6f6d71296532 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:06.740432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b807310-dba8-4f60-a430-6f6d71296532 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:06.741119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b807310-dba8-4f60-a430-6f6d71296532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:06.741418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b807310-dba8-4f60-a430-6f6d71296532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:06.743150 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b807310-dba8-4f60-a430-6f6d71296532 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:06.763400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 272/549] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:06.769160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef663a1-b13f-4a64-87e1-eb1239a59fd2 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:06.915011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef663a1-b13f-4a64-87e1-eb1239a59fd2 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:06.949420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b807310-dba8-4f60-a430-6f6d71296532 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:06.967777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e203a5d-2184-4b89-a7dd-2c8bd1d73189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:06.968411 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e203a5d-2184-4b89-a7dd-2c8bd1d73189 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:06.968689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e203a5d-2184-4b89-a7dd-2c8bd1d73189 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:06.968983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e203a5d-2184-4b89-a7dd-2c8bd1d73189 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:06.986188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e203a5d-2184-4b89-a7dd-2c8bd1d73189 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:06.987655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e203a5d-2184-4b89-a7dd-2c8bd1d73189 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:06.995141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-7e203a5d-2184-4b89-a7dd-2c8bd1d73189 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:07.010745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:07.011678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:07.012545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:07.021005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 278/550] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:07.021996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e203a5d-2184-4b89-a7dd-2c8bd1d73189 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:07.076246 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:07.097625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b807310-dba8-4f60-a430-6f6d71296532 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:07.139969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d246e3c-f3f0-4ddb-aa82-9fdf7f89787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:07.140611 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246e3c-f3f0-4ddb-aa82-9fdf7f89787a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.140912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246e3c-f3f0-4ddb-aa82-9fdf7f89787a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.141183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246e3c-f3f0-4ddb-aa82-9fdf7f89787a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.154325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d246e3c-f3f0-4ddb-aa82-9fdf7f89787a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6d246e3c-f3f0-4ddb-aa82-9fdf7f89787a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:07.159741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:07.162068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:07.164053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 273/551] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:07.168888 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d246e3c-f3f0-4ddb-aa82-9fdf7f89787a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:07.283577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e203a5d-2184-4b89-a7dd-2c8bd1d73189 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:07.287755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d246e3c-f3f0-4ddb-aa82-9fdf7f89787a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:07.330946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03214d47-b9aa-47ec-a376-79320dd43a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:07.331825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03214d47-b9aa-47ec-a376-79320dd43a75 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.332105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03214d47-b9aa-47ec-a376-79320dd43a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.332382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03214d47-b9aa-47ec-a376-79320dd43a75 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.333719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03214d47-b9aa-47ec-a376-79320dd43a75 None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:07.334031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03214d47-b9aa-47ec-a376-79320dd43a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:07.336046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03214d47-b9aa-47ec-a376-79320dd43a75 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:07.341285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1759b81-c69e-4252-9cb2-52a68375852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:07.341878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1759b81-c69e-4252-9cb2-52a68375852b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.342149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1759b81-c69e-4252-9cb2-52a68375852b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.342410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1759b81-c69e-4252-9cb2-52a68375852b None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.343126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1759b81-c69e-4252-9cb2-52a68375852b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:07.343422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1759b81-c69e-4252-9cb2-52a68375852b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:07.344898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1759b81-c69e-4252-9cb2-52a68375852b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:07.345853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 274/552] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:07.347313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd63f3-f595-4647-b812-dfd59afa8222 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.349155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd63f3-f595-4647-b812-dfd59afa8222 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.349430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd63f3-f595-4647-b812-dfd59afa8222 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.356883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 275/553] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:07.357933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd63f3-f595-4647-b812-dfd59afa8222 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:07.363315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:07.371827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 279/554] 10.222.0.84 () {56 vars in 1135 bytes} [Wed Aug 24 11:12:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:07.376243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d168df-6c78-4726-95cc-240e135f26c4 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.376243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d168df-6c78-4726-95cc-240e135f26c4 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.376243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d168df-6c78-4726-95cc-240e135f26c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.393068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c4d168df-6c78-4726-95cc-240e135f26c4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:07.394372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c4d168df-6c78-4726-95cc-240e135f26c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:07.456492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:07.471573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 280/555] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:07.472696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d168df-6c78-4726-95cc-240e135f26c4 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:07.491877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd63f3-f595-4647-b812-dfd59afa8222 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:07.581706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac0b08-ff83-43ae-9143-a6bf74cce5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:07.582442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac0b08-ff83-43ae-9143-a6bf74cce5ce None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.582725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac0b08-ff83-43ae-9143-a6bf74cce5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.582999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac0b08-ff83-43ae-9143-a6bf74cce5ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.591267 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d168df-6c78-4726-95cc-240e135f26c4 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:07.608529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac0b08-ff83-43ae-9143-a6bf74cce5ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:07.610154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac0b08-ff83-43ae-9143-a6bf74cce5ce None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:07.633569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a2609-894d-45fa-a27c-a7da5128a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:07.634234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a2609-894d-45fa-a27c-a7da5128a1ef None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.636385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a2609-894d-45fa-a27c-a7da5128a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.636831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a2609-894d-45fa-a27c-a7da5128a1ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-76ac0b08-ff83-43ae-9143-a6bf74cce5ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:07.645945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:07.662096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:07.665559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:07.665559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:07.665559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:07.665559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 276/556] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:07.665559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac0b08-ff83-43ae-9143-a6bf74cce5ce None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:07.701368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:07.716735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a2609-894d-45fa-a27c-a7da5128a1ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:07.718221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a2609-894d-45fa-a27c-a7da5128a1ef None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:07.722446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:07.771002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac0b08-ff83-43ae-9143-a6bf74cce5ce None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:07.809188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96d289-9ca1-4389-8f31-83d5f8fbf9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:07.809657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96d289-9ca1-4389-8f31-83d5f8fbf9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.809919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96d289-9ca1-4389-8f31-83d5f8fbf9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.810181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96d289-9ca1-4389-8f31-83d5f8fbf9a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.810827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96d289-9ca1-4389-8f31-83d5f8fbf9a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:07.811126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96d289-9ca1-4389-8f31-83d5f8fbf9a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:07.812555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96d289-9ca1-4389-8f31-83d5f8fbf9a7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:07.825339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 277/557] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:07.826560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6471470-c62e-49e7-8009-07a7131579a8 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.827132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6471470-c62e-49e7-8009-07a7131579a8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.827132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6471470-c62e-49e7-8009-07a7131579a8 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.828577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 278/558] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:07.830270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6471470-c62e-49e7-8009-07a7131579a8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:07.839423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 281/559] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:07.841243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48334dd-5371-49b5-a17d-2d8956ab8bb9 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:07.841717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48334dd-5371-49b5-a17d-2d8956ab8bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:07.842179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48334dd-5371-49b5-a17d-2d8956ab8bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:07.861216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e48334dd-5371-49b5-a17d-2d8956ab8bb9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:07.862372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e48334dd-5371-49b5-a17d-2d8956ab8bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:07.937897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 282/560] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:07.938966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48334dd-5371-49b5-a17d-2d8956ab8bb9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:07.960206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:07.961236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6471470-c62e-49e7-8009-07a7131579a8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:07.986490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:08.010260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bb25d-0d91-4077-aebc-7c4c6e55abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.011225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bb25d-0d91-4077-aebc-7c4c6e55abe0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.011722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bb25d-0d91-4077-aebc-7c4c6e55abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.012247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bb25d-0d91-4077-aebc-7c4c6e55abe0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.020736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bb25d-0d91-4077-aebc-7c4c6e55abe0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.022697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bb25d-0d91-4077-aebc-7c4c6e55abe0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3d7bb25d-0d91-4077-aebc-7c4c6e55abe0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:08.036493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:08.037595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:12:08.038488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:08.041657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 279/561] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:08.043061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bb25d-0d91-4077-aebc-7c4c6e55abe0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.049023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48334dd-5371-49b5-a17d-2d8956ab8bb9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.089100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4cf0f-e115-4b71-9134-7c4f88b6cb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.089540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4cf0f-e115-4b71-9134-7c4f88b6cb95 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.089864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4cf0f-e115-4b71-9134-7c4f88b6cb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.090131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4cf0f-e115-4b71-9134-7c4f88b6cb95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.100856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4cf0f-e115-4b71-9134-7c4f88b6cb95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-96b4cf0f-e115-4b71-9134-7c4f88b6cb95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:08.105711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:08.106645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:08.107781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 283/562] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:08.108453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4cf0f-e115-4b71-9134-7c4f88b6cb95 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.168837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bb25d-0d91-4077-aebc-7c4c6e55abe0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.210248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7b763-1b92-4fea-b669-8f767fa1271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.211229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7b763-1b92-4fea-b669-8f767fa1271b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.212000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7b763-1b92-4fea-b669-8f767fa1271b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.212709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7b763-1b92-4fea-b669-8f767fa1271b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.214179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:08.224171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7b763-1b92-4fea-b669-8f767fa1271b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.225656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7b763-1b92-4fea-b669-8f767fa1271b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.240477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4cf0f-e115-4b71-9134-7c4f88b6cb95 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a4c7b763-1b92-4fea-b669-8f767fa1271b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:08.241839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:08.243422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:08.244477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:08.244477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:08.244477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:08.244477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 280/563] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:08.244695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7b763-1b92-4fea-b669-8f767fa1271b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.275592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:08.279986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d1cae-145f-459b-912a-cd75dc7695c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.280850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d1cae-145f-459b-912a-cd75dc7695c5 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.281343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d1cae-145f-459b-912a-cd75dc7695c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.281780 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d1cae-145f-459b-912a-cd75dc7695c5 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.282515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d1cae-145f-459b-912a-cd75dc7695c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:08.282953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d1cae-145f-459b-912a-cd75dc7695c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.284373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d1cae-145f-459b-912a-cd75dc7695c5 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.316616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 284/564] 10.222.0.84 () {56 vars in 1137 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:08.316616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d1cae-145f-459b-912a-cd75dc7695c5 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.367500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7b763-1b92-4fea-b669-8f767fa1271b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.405615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293a79a-2ce2-46ff-adfd-4bbebd0f2d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.406224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293a79a-2ce2-46ff-adfd-4bbebd0f2d4a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.406492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293a79a-2ce2-46ff-adfd-4bbebd0f2d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.406754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293a79a-2ce2-46ff-adfd-4bbebd0f2d4a None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.407412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293a79a-2ce2-46ff-adfd-4bbebd0f2d4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:08.407707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293a79a-2ce2-46ff-adfd-4bbebd0f2d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.409088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293a79a-2ce2-46ff-adfd-4bbebd0f2d4a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.422697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 281/565] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:08.423428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293a79a-2ce2-46ff-adfd-4bbebd0f2d4a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.430339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d1cae-145f-459b-912a-cd75dc7695c5 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.468312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2288df-ee83-4a71-8f1a-abdadcb3f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.469349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2288df-ee83-4a71-8f1a-abdadcb3f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.469632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2288df-ee83-4a71-8f1a-abdadcb3f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.469905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2288df-ee83-4a71-8f1a-abdadcb3f9d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.473014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:08.482894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2288df-ee83-4a71-8f1a-abdadcb3f9d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-5a2288df-ee83-4a71-8f1a-abdadcb3f9d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:08.487655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:08.488703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 285/566] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:08.493736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2288df-ee83-4a71-8f1a-abdadcb3f9d6 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.530518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:08.543241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293a79a-2ce2-46ff-adfd-4bbebd0f2d4a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.578970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eae1a9-94d1-4f00-a386-84023e63fb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.579689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eae1a9-94d1-4f00-a386-84023e63fb67 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.580034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eae1a9-94d1-4f00-a386-84023e63fb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.580401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eae1a9-94d1-4f00-a386-84023e63fb67 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.581261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eae1a9-94d1-4f00-a386-84023e63fb67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:08.581705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eae1a9-94d1-4f00-a386-84023e63fb67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.583134 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eae1a9-94d1-4f00-a386-84023e63fb67 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.595740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 282/567] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:08.600983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eae1a9-94d1-4f00-a386-84023e63fb67 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.603180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2288df-ee83-4a71-8f1a-abdadcb3f9d6 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.639662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47cdf2-7943-44d0-a55f-99ad31ed9b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.640287 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47cdf2-7943-44d0-a55f-99ad31ed9b0f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.640563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47cdf2-7943-44d0-a55f-99ad31ed9b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.640856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47cdf2-7943-44d0-a55f-99ad31ed9b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.658199 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47cdf2-7943-44d0-a55f-99ad31ed9b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.660128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47cdf2-7943-44d0-a55f-99ad31ed9b0f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-dc47cdf2-7943-44d0-a55f-99ad31ed9b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:08.676737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:08.677931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:08.679011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:08.679011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:08.679011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:08.679011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 286/568] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:08.679011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47cdf2-7943-44d0-a55f-99ad31ed9b0f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.714851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eae1a9-94d1-4f00-a386-84023e63fb67 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.728769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:08.749082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e76052-a6b5-4135-a00f-1148aae73e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.749631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e76052-a6b5-4135-a00f-1148aae73e47 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.749904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e76052-a6b5-4135-a00f-1148aae73e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.750171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e76052-a6b5-4135-a00f-1148aae73e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.760878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e76052-a6b5-4135-a00f-1148aae73e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.762404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e76052-a6b5-4135-a00f-1148aae73e47 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f8e76052-a6b5-4135-a00f-1148aae73e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:08.777764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:08.778856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:08.779716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:08.779716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:08.779716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:08.779716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 283/569] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:08.779716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e76052-a6b5-4135-a00f-1148aae73e47 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.782366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:08.786627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47cdf2-7943-44d0-a55f-99ad31ed9b0f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.820521 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51843f4-8efa-4e00-9efc-d793f9efbba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.821698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51843f4-8efa-4e00-9efc-d793f9efbba7 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.821974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51843f4-8efa-4e00-9efc-d793f9efbba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.822383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51843f4-8efa-4e00-9efc-d793f9efbba7 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.823059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51843f4-8efa-4e00-9efc-d793f9efbba7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:08.823340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51843f4-8efa-4e00-9efc-d793f9efbba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.824696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51843f4-8efa-4e00-9efc-d793f9efbba7 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.833172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 287/570] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:08.834457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2c53b-1dd2-4875-b9a3-39fc9320d6c6 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.834741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2c53b-1dd2-4875-b9a3-39fc9320d6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.835018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2c53b-1dd2-4875-b9a3-39fc9320d6c6 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.836259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 288/571] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:08.840986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2c53b-1dd2-4875-b9a3-39fc9320d6c6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.886394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e76052-a6b5-4135-a00f-1148aae73e47 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.920104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fb9fd-c943-4419-8eb6-8735e6b55b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.920848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fb9fd-c943-4419-8eb6-8735e6b55b0c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.921131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fb9fd-c943-4419-8eb6-8735e6b55b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.921634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fb9fd-c943-4419-8eb6-8735e6b55b0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.927884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138fb9fd-c943-4419-8eb6-8735e6b55b0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.929350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138fb9fd-c943-4419-8eb6-8735e6b55b0c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.939763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2c53b-1dd2-4875-b9a3-39fc9320d6c6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-138fb9fd-c943-4419-8eb6-8735e6b55b0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:08.940824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:08.941745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 284/572] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:08.942633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fb9fd-c943-4419-8eb6-8735e6b55b0c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:08.954791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:08.978085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb1ef1-6e2b-4efd-99a7-c4fb743a7f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:08.978673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb1ef1-6e2b-4efd-99a7-c4fb743a7f55 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:08.978945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb1ef1-6e2b-4efd-99a7-c4fb743a7f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:08.979208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb1ef1-6e2b-4efd-99a7-c4fb743a7f55 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:08.979840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb1ef1-6e2b-4efd-99a7-c4fb743a7f55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:08.980137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb1ef1-6e2b-4efd-99a7-c4fb743a7f55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:08.981530 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb1ef1-6e2b-4efd-99a7-c4fb743a7f55 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:08.993714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 289/573] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:08.994579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb1ef1-6e2b-4efd-99a7-c4fb743a7f55 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.007061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:09.045291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fb9fd-c943-4419-8eb6-8735e6b55b0c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.079080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c6b53-b7af-4953-b6bd-baa4bff7a4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.079673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c6b53-b7af-4953-b6bd-baa4bff7a4bc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.079945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c6b53-b7af-4953-b6bd-baa4bff7a4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.080211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c6b53-b7af-4953-b6bd-baa4bff7a4bc None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.080871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c6b53-b7af-4953-b6bd-baa4bff7a4bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:09.081167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c6b53-b7af-4953-b6bd-baa4bff7a4bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.082509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c6b53-b7af-4953-b6bd-baa4bff7a4bc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.092625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb1ef1-6e2b-4efd-99a7-c4fb743a7f55 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.094548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 285/574] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:09.095715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57392a4-5991-4b19-b925-25c6387bafde None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.095994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57392a4-5991-4b19-b925-25c6387bafde None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.096265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57392a4-5991-4b19-b925-25c6387bafde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.114223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c57392a4-5991-4b19-b925-25c6387bafde None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:09.115412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c57392a4-5991-4b19-b925-25c6387bafde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:09.134263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725bd0d-76b8-4a1a-90da-19fe9cf4bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.134871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725bd0d-76b8-4a1a-90da-19fe9cf4bf46 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.135143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725bd0d-76b8-4a1a-90da-19fe9cf4bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.135403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725bd0d-76b8-4a1a-90da-19fe9cf4bf46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.145052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725bd0d-76b8-4a1a-90da-19fe9cf4bf46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.146484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725bd0d-76b8-4a1a-90da-19fe9cf4bf46 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-5725bd0d-76b8-4a1a-90da-19fe9cf4bf46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:09.160485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:09.161648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:09.162878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:09.162878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:09.162878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:09.162878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 290/575] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:09.162878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725bd0d-76b8-4a1a-90da-19fe9cf4bf46 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.184366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 286/576] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:09.185632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57392a4-5991-4b19-b925-25c6387bafde None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.187785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:09.250372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:09.267185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725bd0d-76b8-4a1a-90da-19fe9cf4bf46 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.301765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57392a4-5991-4b19-b925-25c6387bafde None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.301956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c059ccc-0291-4d11-850e-369cd4c31c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.302486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c059ccc-0291-4d11-850e-369cd4c31c21 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.302759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c059ccc-0291-4d11-850e-369cd4c31c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.303023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c059ccc-0291-4d11-850e-369cd4c31c21 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.303706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c059ccc-0291-4d11-850e-369cd4c31c21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:09.304001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c059ccc-0291-4d11-850e-369cd4c31c21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.306000 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c059ccc-0291-4d11-850e-369cd4c31c21 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.317966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 291/577] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:09.318623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c059ccc-0291-4d11-850e-369cd4c31c21 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.344646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef038a-aea5-4457-a0cd-2db0baae6edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.345294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef038a-aea5-4457-a0cd-2db0baae6edb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.345565 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef038a-aea5-4457-a0cd-2db0baae6edb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.345953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef038a-aea5-4457-a0cd-2db0baae6edb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.352326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef038a-aea5-4457-a0cd-2db0baae6edb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.353775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef038a-aea5-4457-a0cd-2db0baae6edb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-ebef038a-aea5-4457-a0cd-2db0baae6edb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:09.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:09.365930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 287/578] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:09.366790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef038a-aea5-4457-a0cd-2db0baae6edb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.410354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:09.429499 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c059ccc-0291-4d11-850e-369cd4c31c21 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.465418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd19987-96b3-43c7-ad8e-67253124d1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.466178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd19987-96b3-43c7-ad8e-67253124d1ac None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.466524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd19987-96b3-43c7-ad8e-67253124d1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.466857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd19987-96b3-43c7-ad8e-67253124d1ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.473812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd19987-96b3-43c7-ad8e-67253124d1ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.475270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd19987-96b3-43c7-ad8e-67253124d1ac None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.479277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef038a-aea5-4457-a0cd-2db0baae6edb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-5fd19987-96b3-43c7-ad8e-67253124d1ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:09.487661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:09.488715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 292/579] 10.222.0.84 () {56 vars in 1093 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:09.489727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd19987-96b3-43c7-ad8e-67253124d1ac None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.498335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:09.519949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4541f-4800-40b4-95b0-17ecd13f8469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.520485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4541f-4800-40b4-95b0-17ecd13f8469 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.520759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4541f-4800-40b4-95b0-17ecd13f8469 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.521091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4541f-4800-40b4-95b0-17ecd13f8469 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.521723 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4541f-4800-40b4-95b0-17ecd13f8469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:09.522037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4541f-4800-40b4-95b0-17ecd13f8469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.523418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4541f-4800-40b4-95b0-17ecd13f8469 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.534603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 288/580] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:09.535383 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4541f-4800-40b4-95b0-17ecd13f8469 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.591826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd19987-96b3-43c7-ad8e-67253124d1ac None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.626466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42166f16-afb8-43e9-98ec-f159562d93f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.627060 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42166f16-afb8-43e9-98ec-f159562d93f0 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.627340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42166f16-afb8-43e9-98ec-f159562d93f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.627614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42166f16-afb8-43e9-98ec-f159562d93f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.628325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42166f16-afb8-43e9-98ec-f159562d93f0 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=1ba54c2312b348b08e0faae745747e7d) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:09.628628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42166f16-afb8-43e9-98ec-f159562d93f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.630077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42166f16-afb8-43e9-98ec-f159562d93f0 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.635827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 293/581] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/role_assignments?user.id=1ba54c2312b348b08e0faae745747e7d&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:09.636598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42166f16-afb8-43e9-98ec-f159562d93f0 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.644104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4541f-4800-40b4-95b0-17ecd13f8469 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.645841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:09.691253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e6cd2-92c0-4a6f-8ff1-6e08120989fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.691874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e6cd2-92c0-4a6f-8ff1-6e08120989fb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.692153 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e6cd2-92c0-4a6f-8ff1-6e08120989fb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.692425 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e6cd2-92c0-4a6f-8ff1-6e08120989fb None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.693154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e6cd2-92c0-4a6f-8ff1-6e08120989fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:09.693480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e6cd2-92c0-4a6f-8ff1-6e08120989fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.694903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e6cd2-92c0-4a6f-8ff1-6e08120989fb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.711554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 289/582] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:09.713310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e6cd2-92c0-4a6f-8ff1-6e08120989fb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.749511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:09.768290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42166f16-afb8-43e9-98ec-f159562d93f0 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.805117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2155-dd4a-473a-bde7-70d118d0a2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.806268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2155-dd4a-473a-bde7-70d118d0a2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.806588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2155-dd4a-473a-bde7-70d118d0a2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.806802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2155-dd4a-473a-bde7-70d118d0a2a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.817105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d2155-dd4a-473a-bde7-70d118d0a2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-6e6d2155-dd4a-473a-bde7-70d118d0a2a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:09.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:09.823291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:09.824123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 294/583] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:09.824845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2155-dd4a-473a-bde7-70d118d0a2a6 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.830264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e6cd2-92c0-4a6f-8ff1-6e08120989fb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.866672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81594c-14c1-4326-904f-62827460a65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.867277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81594c-14c1-4326-904f-62827460a65c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.867556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81594c-14c1-4326-904f-62827460a65c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.868402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81594c-14c1-4326-904f-62827460a65c None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.869799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81594c-14c1-4326-904f-62827460a65c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:09.870107 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81594c-14c1-4326-904f-62827460a65c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.871530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81594c-14c1-4326-904f-62827460a65c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.891393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 290/584] 10.222.0.84 () {56 vars in 1133 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:09.892946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81594c-14c1-4326-904f-62827460a65c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.896502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:09.935625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2155-dd4a-473a-bde7-70d118d0a2a6 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:09.971377 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96816eb1-50c0-4cc2-93a6-ff783329af79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:09.972015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96816eb1-50c0-4cc2-93a6-ff783329af79 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:09.972261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96816eb1-50c0-4cc2-93a6-ff783329af79 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:09.972526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96816eb1-50c0-4cc2-93a6-ff783329af79 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:09.980575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96816eb1-50c0-4cc2-93a6-ff783329af79 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:09.982318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96816eb1-50c0-4cc2-93a6-ff783329af79 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-96816eb1-50c0-4cc2-93a6-ff783329af79 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:09.993093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:09.994142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 295/585] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:09.995055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96816eb1-50c0-4cc2-93a6-ff783329af79 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:09.997899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:10.009966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81594c-14c1-4326-904f-62827460a65c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.047443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eee2f0-ddae-4ec7-9290-93f38693fca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.048102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eee2f0-ddae-4ec7-9290-93f38693fca0 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.048378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eee2f0-ddae-4ec7-9290-93f38693fca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.048652 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eee2f0-ddae-4ec7-9290-93f38693fca0 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/75ac781cf2d84d95a789724e74e12acf/roles/69cd1c433d70467a91f4a9f47382a412` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.064101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eee2f0-ddae-4ec7-9290-93f38693fca0 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=75ac781cf2d84d95a789724e74e12acf, role_id=69cd1c433d70467a91f4a9f47382a412)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.065590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eee2f0-ddae-4ec7-9290-93f38693fca0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.102980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96816eb1-50c0-4cc2-93a6-ff783329af79 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.127796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:10.144825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 291/586] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:12:09 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/75ac781cf2d84d95a789724e74e12acf/roles/69cd1c433d70467a91f4a9f47382a412 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:10.144825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eee2f0-ddae-4ec7-9290-93f38693fca0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:10.147877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62313c96-8f39-490f-a62d-51c1596df0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.150114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62313c96-8f39-490f-a62d-51c1596df0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.150114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62313c96-8f39-490f-a62d-51c1596df0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.150578 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62313c96-8f39-490f-a62d-51c1596df0a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.151565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62313c96-8f39-490f-a62d-51c1596df0a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:10.151876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62313c96-8f39-490f-a62d-51c1596df0a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.153575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62313c96-8f39-490f-a62d-51c1596df0a1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.166631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 296/587] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:10.168557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62313c96-8f39-490f-a62d-51c1596df0a1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:10.268990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eee2f0-ddae-4ec7-9290-93f38693fca0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.314584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a74be-eddb-48a4-be16-e4237457fd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.315255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a74be-eddb-48a4-be16-e4237457fd5d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.315596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a74be-eddb-48a4-be16-e4237457fd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.315801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a74be-eddb-48a4-be16-e4237457fd5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.317323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62313c96-8f39-490f-a62d-51c1596df0a1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.326961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:10.330087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a74be-eddb-48a4-be16-e4237457fd5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.331712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a74be-eddb-48a4-be16-e4237457fd5d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-131a74be-eddb-48a4-be16-e4237457fd5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:10.349617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:10.350682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:10.351765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:10.351765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:10.351765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:10.355334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 292/588] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:10.356219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a74be-eddb-48a4-be16-e4237457fd5d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:10.376758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732b338-a7a9-4de8-9e5f-e73e8e851179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.376758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732b338-a7a9-4de8-9e5f-e73e8e851179 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.376758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732b338-a7a9-4de8-9e5f-e73e8e851179 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.376758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732b338-a7a9-4de8-9e5f-e73e8e851179 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.377151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732b338-a7a9-4de8-9e5f-e73e8e851179 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:10.377151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732b338-a7a9-4de8-9e5f-e73e8e851179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.377151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732b338-a7a9-4de8-9e5f-e73e8e851179 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.398869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 297/589] 10.222.0.84 () {56 vars in 1137 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:10.404604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732b338-a7a9-4de8-9e5f-e73e8e851179 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:10.460038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:10.479481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a74be-eddb-48a4-be16-e4237457fd5d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.524730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732b338-a7a9-4de8-9e5f-e73e8e851179 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.529957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0216-6d0c-4e9a-8c1e-f21d4cbe0d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.530590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0216-6d0c-4e9a-8c1e-f21d4cbe0d59 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.530863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0216-6d0c-4e9a-8c1e-f21d4cbe0d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.531131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0216-6d0c-4e9a-8c1e-f21d4cbe0d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.545894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0216-6d0c-4e9a-8c1e-f21d4cbe0d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.547349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0216-6d0c-4e9a-8c1e-f21d4cbe0d59 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.562099 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7fa42-2dce-452e-9f80-29a69b429202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.562843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7fa42-2dce-452e-9f80-29a69b429202 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.563230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7fa42-2dce-452e-9f80-29a69b429202 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.563604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7fa42-2dce-452e-9f80-29a69b429202 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.564388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7fa42-2dce-452e-9f80-29a69b429202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:10.564868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7fa42-2dce-452e-9f80-29a69b429202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-b0fe0216-6d0c-4e9a-8c1e-f21d4cbe0d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:10.565858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:10.567001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:10.568076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:10.568076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:10.568076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:10.568076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 293/590] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:10.568076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0216-6d0c-4e9a-8c1e-f21d4cbe0d59 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:10.568734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7fa42-2dce-452e-9f80-29a69b429202 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.582749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 298/591] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:10.583775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7fa42-2dce-452e-9f80-29a69b429202 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:10.632016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:10.712786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0216-6d0c-4e9a-8c1e-f21d4cbe0d59 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.714032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7fa42-2dce-452e-9f80-29a69b429202 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.733250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:10.759737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78031384-4015-4ebb-8777-4d5ea32a1fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.760392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78031384-4015-4ebb-8777-4d5ea32a1fdc None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.760771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78031384-4015-4ebb-8777-4d5ea32a1fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.761282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78031384-4015-4ebb-8777-4d5ea32a1fdc None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.762086 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78031384-4015-4ebb-8777-4d5ea32a1fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:10.762467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78031384-4015-4ebb-8777-4d5ea32a1fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.763961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78031384-4015-4ebb-8777-4d5ea32a1fdc None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.765546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08724f-b084-43ab-99b2-4c4373b5a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.766856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08724f-b084-43ab-99b2-4c4373b5a8ad None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.767340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08724f-b084-43ab-99b2-4c4373b5a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.767795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08724f-b084-43ab-99b2-4c4373b5a8ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.778534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 299/592] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:10.779412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78031384-4015-4ebb-8777-4d5ea32a1fdc None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:10.784241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08724f-b084-43ab-99b2-4c4373b5a8ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.786131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08724f-b084-43ab-99b2-4c4373b5a8ad None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6c08724f-b084-43ab-99b2-4c4373b5a8ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:10.807009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:10.807971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:10.808891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:10.808891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:10.808891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:10.810593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 294/593] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:10.811176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08724f-b084-43ab-99b2-4c4373b5a8ad None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:10.911093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78031384-4015-4ebb-8777-4d5ea32a1fdc None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.921652 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:10.929730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08724f-b084-43ab-99b2-4c4373b5a8ad None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:10.952191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06079f-018b-447f-a992-1160efb73e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.953691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06079f-018b-447f-a992-1160efb73e72 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.954157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06079f-018b-447f-a992-1160efb73e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.954607 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06079f-018b-447f-a992-1160efb73e72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.965884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc0e8c-cca8-44f9-8d58-5b6a30f9f5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:10.966318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06079f-018b-447f-a992-1160efb73e72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.966665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0e8c-cca8-44f9-8d58-5b6a30f9f5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:10.967008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0e8c-cca8-44f9-8d58-5b6a30f9f5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:10.967345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc0e8c-cca8-44f9-8d58-5b6a30f9f5b7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:10.968715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06079f-018b-447f-a992-1160efb73e72 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.975436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc0e8c-cca8-44f9-8d58-5b6a30f9f5b7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:10.977834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc0e8c-cca8-44f9-8d58-5b6a30f9f5b7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-0d06079f-018b-447f-a992-1160efb73e72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:10.984274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:10.985660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:10.986728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:10.986728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:10.986728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:10.986728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:10.986728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:10.986728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:10.986728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:10.986728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e9cc0e8c-cca8-44f9-8d58-5b6a30f9f5b7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:10.993396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:10.994573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:12:10.995734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:10.996677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 300/594] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:10.997290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 295/595] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:10.997836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06079f-018b-447f-a992-1160efb73e72 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:11.005209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc0e8c-cca8-44f9-8d58-5b6a30f9f5b7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:11.081189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:11.127345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06079f-018b-447f-a992-1160efb73e72 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:11.132788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc0e8c-cca8-44f9-8d58-5b6a30f9f5b7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:11.172253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d968b4-5311-4254-9ebb-382df39fd6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:11.173717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d968b4-5311-4254-9ebb-382df39fd6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.174143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d968b4-5311-4254-9ebb-382df39fd6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.174830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d968b4-5311-4254-9ebb-382df39fd6e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.175803 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d968b4-5311-4254-9ebb-382df39fd6e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:11.176335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d968b4-5311-4254-9ebb-382df39fd6e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:11.178272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d968b4-5311-4254-9ebb-382df39fd6e0 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:11.178799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64689df0-dc13-4fcb-bc55-badbe05b6064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:11.179544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64689df0-dc13-4fcb-bc55-badbe05b6064 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.179915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64689df0-dc13-4fcb-bc55-badbe05b6064 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.180480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64689df0-dc13-4fcb-bc55-badbe05b6064 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.181142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64689df0-dc13-4fcb-bc55-badbe05b6064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:11.182124 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64689df0-dc13-4fcb-bc55-badbe05b6064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:11.184331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64689df0-dc13-4fcb-bc55-badbe05b6064 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:11.193308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 301/596] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:11.194348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d968b4-5311-4254-9ebb-382df39fd6e0 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:11.207804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 296/597] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:11.207804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64689df0-dc13-4fcb-bc55-badbe05b6064 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:11.227547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:11.312879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:11.318697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64689df0-dc13-4fcb-bc55-badbe05b6064 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:11.329866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d968b4-5311-4254-9ebb-382df39fd6e0 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:11.364542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca953-2b3b-4007-b31d-e932a6e41992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:11.365424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca953-2b3b-4007-b31d-e932a6e41992 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.366059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca953-2b3b-4007-b31d-e932a6e41992 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.366404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca953-2b3b-4007-b31d-e932a6e41992 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.379682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ca953-2b3b-4007-b31d-e932a6e41992 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:11.382098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ca953-2b3b-4007-b31d-e932a6e41992 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:11.388176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7bf4f-74c0-48f9-924e-d12fc4d747da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:11.389065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7bf4f-74c0-48f9-924e-d12fc4d747da None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.389550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7bf4f-74c0-48f9-924e-d12fc4d747da None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.390036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7bf4f-74c0-48f9-924e-d12fc4d747da None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.391156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7bf4f-74c0-48f9-924e-d12fc4d747da None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:11.391672 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7bf4f-74c0-48f9-924e-d12fc4d747da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:11.393436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7bf4f-74c0-48f9-924e-d12fc4d747da None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-ab8ca953-2b3b-4007-b31d-e932a6e41992 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:11.394700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:11.395663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:11.396539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:11.396539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:11.396539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:11.396539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:11.396539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:11.396539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:11.396539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:11.396539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:11.400949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 297/598] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:11.402091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca953-2b3b-4007-b31d-e932a6e41992 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:11.433170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 302/599] 10.222.0.84 () {56 vars in 1139 bytes} [Wed Aug 24 11:12:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:11.433170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7bf4f-74c0-48f9-924e-d12fc4d747da None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:11.501797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:11.525848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca953-2b3b-4007-b31d-e932a6e41992 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:11.566766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7bf4f-74c0-48f9-924e-d12fc4d747da None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:11.573088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e7f82-4b73-4349-98ce-f5503305766e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:11.574038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e7f82-4b73-4349-98ce-f5503305766e None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.574526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e7f82-4b73-4349-98ce-f5503305766e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.575001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e7f82-4b73-4349-98ce-f5503305766e None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/f58a559951c94d68951da78c82341a00/roles/69cd1c433d70467a91f4a9f47382a412` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.594371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e7f82-4b73-4349-98ce-f5503305766e None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=f58a559951c94d68951da78c82341a00, role_id=69cd1c433d70467a91f4a9f47382a412)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:11.595796 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e7f82-4b73-4349-98ce-f5503305766e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:11.606241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7f780-fba7-424e-832d-8cb5c99176c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:11.606905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7f780-fba7-424e-832d-8cb5c99176c8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.612444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7f780-fba7-424e-832d-8cb5c99176c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.612444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7f780-fba7-424e-832d-8cb5c99176c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.612444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7f780-fba7-424e-832d-8cb5c99176c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:11.612444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7f780-fba7-424e-832d-8cb5c99176c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:11.612444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7f780-fba7-424e-832d-8cb5c99176c8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:11.622184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 303/600] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:11.622989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7f780-fba7-424e-832d-8cb5c99176c8 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:11.627275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:11.666369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 298/601] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:12:11 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/f58a559951c94d68951da78c82341a00/roles/69cd1c433d70467a91f4a9f47382a412 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:11.667697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1e4c6-ef4a-4319-ac37-7aad5fd520df None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.667982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1e4c6-ef4a-4319-ac37-7aad5fd520df None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.668251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1e4c6-ef4a-4319-ac37-7aad5fd520df None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.669495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 299/602] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:11.674119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1e4c6-ef4a-4319-ac37-7aad5fd520df None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:11.773776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7f780-fba7-424e-832d-8cb5c99176c8 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:11.800929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1e4c6-ef4a-4319-ac37-7aad5fd520df None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:11.827177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad58fae-0afd-4c2b-8bf0-5e3c4d6fd717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:11.827838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad58fae-0afd-4c2b-8bf0-5e3c4d6fd717 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.828115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad58fae-0afd-4c2b-8bf0-5e3c4d6fd717 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.828381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad58fae-0afd-4c2b-8bf0-5e3c4d6fd717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.838177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:11.844246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad58fae-0afd-4c2b-8bf0-5e3c4d6fd717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:11.849594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad58fae-0afd-4c2b-8bf0-5e3c4d6fd717 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:11.849711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46202f84-d3a7-4223-b61e-8c119ba4e273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:11.849711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46202f84-d3a7-4223-b61e-8c119ba4e273 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.849711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46202f84-d3a7-4223-b61e-8c119ba4e273 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.849927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46202f84-d3a7-4223-b61e-8c119ba4e273 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/d5ce16d5ea8240debfcb69ac7b1f71c0/roles/69cd1c433d70467a91f4a9f47382a412` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-2ad58fae-0afd-4c2b-8bf0-5e3c4d6fd717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:11.863037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:11.864015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:11.864959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:11.864959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:11.864959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:11.864959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 304/603] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:11.865618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c2fb-1165-43bf-8d60-14e2307d4dbb None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:11.865618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c2fb-1165-43bf-8d60-14e2307d4dbb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:11.865865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726c2fb-1165-43bf-8d60-14e2307d4dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:11.872487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46202f84-d3a7-4223-b61e-8c119ba4e273 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=d5ce16d5ea8240debfcb69ac7b1f71c0, role_id=69cd1c433d70467a91f4a9f47382a412)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:11.874312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46202f84-d3a7-4223-b61e-8c119ba4e273 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:11.890099 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b726c2fb-1165-43bf-8d60-14e2307d4dbb None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:11.891300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b726c2fb-1165-43bf-8d60-14e2307d4dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:11.947431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:11.964512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 305/604] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:11.965402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c2fb-1165-43bf-8d60-14e2307d4dbb None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:11.978514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 300/605] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:12:11 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/d5ce16d5ea8240debfcb69ac7b1f71c0/roles/69cd1c433d70467a91f4a9f47382a412 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:12.100943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726c2fb-1165-43bf-8d60-14e2307d4dbb None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:12.139711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e1f37-1d0e-41de-86a5-3e1854434dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:12.140494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e1f37-1d0e-41de-86a5-3e1854434dcf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:12.140932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e1f37-1d0e-41de-86a5-3e1854434dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:12.141293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e1f37-1d0e-41de-86a5-3e1854434dcf None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:12.142043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e1f37-1d0e-41de-86a5-3e1854434dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:12.142438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e1f37-1d0e-41de-86a5-3e1854434dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:12.143923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e1f37-1d0e-41de-86a5-3e1854434dcf None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:12.157560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 306/606] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:12.162835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46202f84-d3a7-4223-b61e-8c119ba4e273 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:12.280368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46202f84-d3a7-4223-b61e-8c119ba4e273 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:12.296925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e1f37-1d0e-41de-86a5-3e1854434dcf None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:12.348864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584baee9-15bc-4ec2-b29e-c22103e8356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:12.350184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584baee9-15bc-4ec2-b29e-c22103e8356b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:12.350889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584baee9-15bc-4ec2-b29e-c22103e8356b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:12.351390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584baee9-15bc-4ec2-b29e-c22103e8356b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:12.353891 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:12.361496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584baee9-15bc-4ec2-b29e-c22103e8356b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:12.363887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584baee9-15bc-4ec2-b29e-c22103e8356b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-584baee9-15bc-4ec2-b29e-c22103e8356b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:12.379604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:12.380638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:12.381722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:12.381722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:12.381722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:12.381722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:12.381722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:12.381722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:12.381722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:12.381722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:12.382638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 301/607] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:12.384587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4a350-ce8e-4c65-bad8-b3bc5794a5ab None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:12.385032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4a350-ce8e-4c65-bad8-b3bc5794a5ab None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:12.385308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4a350-ce8e-4c65-bad8-b3bc5794a5ab None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:12.386803 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 302/608] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:12.392335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4a350-ce8e-4c65-bad8-b3bc5794a5ab None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:12.431169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e1f37-1d0e-41de-86a5-3e1854434dcf None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:12.437225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:12.473603 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7bc6e-7478-44c7-aa7d-749fc4b89fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:12.474714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7bc6e-7478-44c7-aa7d-749fc4b89fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:12.475171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7bc6e-7478-44c7-aa7d-749fc4b89fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:12.475577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7bc6e-7478-44c7-aa7d-749fc4b89fd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:12.490343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7bc6e-7478-44c7-aa7d-749fc4b89fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-e7f7bc6e-7478-44c7-aa7d-749fc4b89fd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:12.498106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:12.499177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:12.500182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:12.502869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 307/609] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:12.504473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f84e7-78c4-43b0-98b2-bb2d3f113a11 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:12.504942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f84e7-78c4-43b0-98b2-bb2d3f113a11 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:12.505399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f84e7-78c4-43b0-98b2-bb2d3f113a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:12.522575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4a350-ce8e-4c65-bad8-b3bc5794a5ab None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:12.527503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ec2f84e7-78c4-43b0-98b2-bb2d3f113a11 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:12.529047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ec2f84e7-78c4-43b0-98b2-bb2d3f113a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:12.570054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8bcf0-61ef-4c5b-b1c1-e3b5771022ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:12.571040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8bcf0-61ef-4c5b-b1c1-e3b5771022ce None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:12.571504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8bcf0-61ef-4c5b-b1c1-e3b5771022ce None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:12.572074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8bcf0-61ef-4c5b-b1c1-e3b5771022ce None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:12.573172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8bcf0-61ef-4c5b-b1c1-e3b5771022ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:12.573747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8bcf0-61ef-4c5b-b1c1-e3b5771022ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:12.575533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8bcf0-61ef-4c5b-b1c1-e3b5771022ce None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:12.588510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 303/610] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:12.589709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8bcf0-61ef-4c5b-b1c1-e3b5771022ce None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:12.620949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 308/611] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:12.622098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f84e7-78c4-43b0-98b2-bb2d3f113a11 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:12.689442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:12.743887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:12.755236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8bcf0-61ef-4c5b-b1c1-e3b5771022ce None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:12.810108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f84e7-78c4-43b0-98b2-bb2d3f113a11 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:12.823913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c883a41-5d75-49b9-82b3-f2487431466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:12.824523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c883a41-5d75-49b9-82b3-f2487431466e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:12.824817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c883a41-5d75-49b9-82b3-f2487431466e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:12.825118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c883a41-5d75-49b9-82b3-f2487431466e None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:12.825781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c883a41-5d75-49b9-82b3-f2487431466e None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:12.826068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c883a41-5d75-49b9-82b3-f2487431466e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:12.827424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c883a41-5d75-49b9-82b3-f2487431466e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:12.842267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 304/612] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:12.842643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c883a41-5d75-49b9-82b3-f2487431466e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:12.874470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f006b9-351e-482d-8631-87970d8bb4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:12.875051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f006b9-351e-482d-8631-87970d8bb4c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:12.875320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f006b9-351e-482d-8631-87970d8bb4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:12.875579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f006b9-351e-482d-8631-87970d8bb4c3 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/372e96bd3361434e9169b783f0c9ea13/roles/69cd1c433d70467a91f4a9f47382a412` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:12.906738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f006b9-351e-482d-8631-87970d8bb4c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=372e96bd3361434e9169b783f0c9ea13, role_id=69cd1c433d70467a91f4a9f47382a412)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:12.908177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f006b9-351e-482d-8631-87970d8bb4c3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:12.974988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c883a41-5d75-49b9-82b3-f2487431466e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:13.004143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 309/613] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:12:12 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/372e96bd3361434e9169b783f0c9ea13/roles/69cd1c433d70467a91f4a9f47382a412 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:13.005628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f006b9-351e-482d-8631-87970d8bb4c3 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:13.016249 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a17308-582b-4f9a-a841-3f93d4b047e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:13.016917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a17308-582b-4f9a-a841-3f93d4b047e2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:13.017355 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a17308-582b-4f9a-a841-3f93d4b047e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:13.017624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a17308-582b-4f9a-a841-3f93d4b047e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:13.031464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a17308-582b-4f9a-a841-3f93d4b047e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:13.033071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a17308-582b-4f9a-a841-3f93d4b047e2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-72a17308-582b-4f9a-a841-3f93d4b047e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:13.052409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:13.053455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:13.054335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:13.054335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:13.054335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:13.056883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 305/614] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:13.062188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a17308-582b-4f9a-a841-3f93d4b047e2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:13.080511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:13.150623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:13.189639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f006b9-351e-482d-8631-87970d8bb4c3 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:13.215794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a17308-582b-4f9a-a841-3f93d4b047e2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:13.260903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b394f-3242-4bfd-9c49-e4919d2a6304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:13.261931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b394f-3242-4bfd-9c49-e4919d2a6304 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:13.262429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b394f-3242-4bfd-9c49-e4919d2a6304 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:13.262918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b394f-3242-4bfd-9c49-e4919d2a6304 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:13.263316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cf12f-976a-4ed6-84b2-e6c1b3be400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:13.264106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cf12f-976a-4ed6-84b2-e6c1b3be400b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:13.264386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cf12f-976a-4ed6-84b2-e6c1b3be400b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:13.264664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cf12f-976a-4ed6-84b2-e6c1b3be400b None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:13.265439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cf12f-976a-4ed6-84b2-e6c1b3be400b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:13.265738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cf12f-976a-4ed6-84b2-e6c1b3be400b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:13.267341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7cf12f-976a-4ed6-84b2-e6c1b3be400b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:13.275992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b394f-3242-4bfd-9c49-e4919d2a6304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:13.280981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 306/615] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-a73b394f-3242-4bfd-9c49-e4919d2a6304 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:13.285983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:13.286898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:13.287708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 310/616] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:13.297984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cf12f-976a-4ed6-84b2-e6c1b3be400b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:13.299277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b394f-3242-4bfd-9c49-e4919d2a6304 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:13.421734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:13.477897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b394f-3242-4bfd-9c49-e4919d2a6304 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:13.499498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cf12f-976a-4ed6-84b2-e6c1b3be400b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:13.550040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1483028-3c1a-487a-a633-c46b841983b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:13.551090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1483028-3c1a-487a-a633-c46b841983b6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:13.551574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1483028-3c1a-487a-a633-c46b841983b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:13.552043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1483028-3c1a-487a-a633-c46b841983b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:13.554671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309468f-ba5c-47fd-9fac-ddbb6db1f9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:13.555936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309468f-ba5c-47fd-9fac-ddbb6db1f9dc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:13.556214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309468f-ba5c-47fd-9fac-ddbb6db1f9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:13.556491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309468f-ba5c-47fd-9fac-ddbb6db1f9dc None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:13.557188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309468f-ba5c-47fd-9fac-ddbb6db1f9dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:13.557491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309468f-ba5c-47fd-9fac-ddbb6db1f9dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:13.559214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309468f-ba5c-47fd-9fac-ddbb6db1f9dc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:13.573171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 307/617] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:13.574297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1483028-3c1a-487a-a633-c46b841983b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:13.574598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309468f-ba5c-47fd-9fac-ddbb6db1f9dc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:13.576193 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1483028-3c1a-487a-a633-c46b841983b6 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-b1483028-3c1a-487a-a633-c46b841983b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:13.610246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:13.611224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:13.612404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 311/618] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:13.619463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1483028-3c1a-487a-a633-c46b841983b6 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:13.632089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:13.714033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309468f-ba5c-47fd-9fac-ddbb6db1f9dc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:13.750000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:13.757714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0e77f-77e8-476f-9c97-a6eda32224ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:13.759288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0e77f-77e8-476f-9c97-a6eda32224ab None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:13.761630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0e77f-77e8-476f-9c97-a6eda32224ab None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:13.763405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0e77f-77e8-476f-9c97-a6eda32224ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:13.789059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0e77f-77e8-476f-9c97-a6eda32224ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:13.792773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0e77f-77e8-476f-9c97-a6eda32224ab None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-cba0e77f-77e8-476f-9c97-a6eda32224ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:13.810435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:13.811615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:13.812559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:13.812559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:13.812559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:13.814363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 308/619] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:13.820423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0e77f-77e8-476f-9c97-a6eda32224ab None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:13.909324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1483028-3c1a-487a-a633-c46b841983b6 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:13.937430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0e77f-77e8-476f-9c97-a6eda32224ab None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:13.946969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986ec70-07f5-4ee3-93ff-9ec9bb9c26c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:13.947518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986ec70-07f5-4ee3-93ff-9ec9bb9c26c6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:13.947787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986ec70-07f5-4ee3-93ff-9ec9bb9c26c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:13.948050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986ec70-07f5-4ee3-93ff-9ec9bb9c26c6 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:13.948767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986ec70-07f5-4ee3-93ff-9ec9bb9c26c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:13.949091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986ec70-07f5-4ee3-93ff-9ec9bb9c26c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:13.950513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986ec70-07f5-4ee3-93ff-9ec9bb9c26c6 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:13.968370 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 312/620] 10.222.0.84 () {56 vars in 1135 bytes} [Wed Aug 24 11:12:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:13.969393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbcf01-b0d2-4bce-8ff8-cdc345e497f8 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:13.969674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbcf01-b0d2-4bce-8ff8-cdc345e497f8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:13.969940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbcf01-b0d2-4bce-8ff8-cdc345e497f8 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:13.975264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 313/621] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:13.976032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbcf01-b0d2-4bce-8ff8-cdc345e497f8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:13.979121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce1511-5854-41e4-a2e8-5c19413a3eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:13.980139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce1511-5854-41e4-a2e8-5c19413a3eba None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:13.980621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce1511-5854-41e4-a2e8-5c19413a3eba None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:13.981133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce1511-5854-41e4-a2e8-5c19413a3eba None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:13.982453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce1511-5854-41e4-a2e8-5c19413a3eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:13.982965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce1511-5854-41e4-a2e8-5c19413a3eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:13.984694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce1511-5854-41e4-a2e8-5c19413a3eba None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:14.000240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 309/622] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:14.006323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a080e34-d2d8-4ab1-8abb-a18ab630a9a6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.006862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a080e34-d2d8-4ab1-8abb-a18ab630a9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.007478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a080e34-d2d8-4ab1-8abb-a18ab630a9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.028881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1a080e34-d2d8-4ab1-8abb-a18ab630a9a6 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:14.030538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1a080e34-d2d8-4ab1-8abb-a18ab630a9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:14.041974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:14.079202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:14.085193 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbcf01-b0d2-4bce-8ff8-cdc345e497f8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:14.109013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 310/623] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:14.109935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a080e34-d2d8-4ab1-8abb-a18ab630a9a6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:14.153988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccba012-8d6d-4d3b-99d1-c67384938781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:14.154678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccba012-8d6d-4d3b-99d1-c67384938781 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.154942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccba012-8d6d-4d3b-99d1-c67384938781 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.155210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccba012-8d6d-4d3b-99d1-c67384938781 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.178995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccba012-8d6d-4d3b-99d1-c67384938781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:14.182808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccba012-8d6d-4d3b-99d1-c67384938781 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-5ccba012-8d6d-4d3b-99d1-c67384938781 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:14.215705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:14.216726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:14.217668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:14.217668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:14.217668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:14.225037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 314/624] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:14.225902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccba012-8d6d-4d3b-99d1-c67384938781 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:14.250593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a080e34-d2d8-4ab1-8abb-a18ab630a9a6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:14.286875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83326e6f-cbc3-4b09-80e8-671697d9e2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:14.287505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83326e6f-cbc3-4b09-80e8-671697d9e2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.287781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83326e6f-cbc3-4b09-80e8-671697d9e2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.288050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83326e6f-cbc3-4b09-80e8-671697d9e2b8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.295584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83326e6f-cbc3-4b09-80e8-671697d9e2b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:14.297050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83326e6f-cbc3-4b09-80e8-671697d9e2b8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-83326e6f-cbc3-4b09-80e8-671697d9e2b8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:14.309028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:14.310008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:14.310938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 311/625] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:14.317379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83326e6f-cbc3-4b09-80e8-671697d9e2b8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:14.381939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:14.428421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccba012-8d6d-4d3b-99d1-c67384938781 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:14.445343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:14.450985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83326e6f-cbc3-4b09-80e8-671697d9e2b8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:14.465785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106d326-0cd0-4ea4-8a68-86f8c0eb4794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:14.466382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d326-0cd0-4ea4-8a68-86f8c0eb4794 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.466659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d326-0cd0-4ea4-8a68-86f8c0eb4794 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.466932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d326-0cd0-4ea4-8a68-86f8c0eb4794 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.467565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106d326-0cd0-4ea4-8a68-86f8c0eb4794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:14.467893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106d326-0cd0-4ea4-8a68-86f8c0eb4794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:14.469294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106d326-0cd0-4ea4-8a68-86f8c0eb4794 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:14.481723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 315/626] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:14.483437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c6f16-4369-4b3e-be17-ba68f555de0c None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.483887 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c6f16-4369-4b3e-be17-ba68f555de0c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.484321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c6f16-4369-4b3e-be17-ba68f555de0c None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.486055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 316/627] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:14.486938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c6f16-4369-4b3e-be17-ba68f555de0c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:14.497074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128d3fc-1b72-4970-b02b-86a46d518d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:14.497687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128d3fc-1b72-4970-b02b-86a46d518d38 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.497965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128d3fc-1b72-4970-b02b-86a46d518d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.498629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128d3fc-1b72-4970-b02b-86a46d518d38 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.499365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128d3fc-1b72-4970-b02b-86a46d518d38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:14.499987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128d3fc-1b72-4970-b02b-86a46d518d38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:14.501436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128d3fc-1b72-4970-b02b-86a46d518d38 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:14.521156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 312/628] 10.222.0.84 () {56 vars in 1137 bytes} [Wed Aug 24 11:12:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:14.521977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128d3fc-1b72-4970-b02b-86a46d518d38 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:14.603722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c6f16-4369-4b3e-be17-ba68f555de0c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:14.620273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:14.630101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128d3fc-1b72-4970-b02b-86a46d518d38 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:14.639048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc326-4ea4-412a-ab3e-e3b1cdd585d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:14.639972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc326-4ea4-412a-ab3e-e3b1cdd585d1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.640244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc326-4ea4-412a-ab3e-e3b1cdd585d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.640561 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc326-4ea4-412a-ab3e-e3b1cdd585d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.655620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc326-4ea4-412a-ab3e-e3b1cdd585d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-ce4fc326-4ea4-412a-ab3e-e3b1cdd585d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:14.660327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:14.661284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:14.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 317/629] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:14.662856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb020b5-e1d0-431d-b5d6-e6948462a7b9 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.662856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb020b5-e1d0-431d-b5d6-e6948462a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.663606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb020b5-e1d0-431d-b5d6-e6948462a7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.680284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c70cfc-f8ef-4251-90a6-523cc0291c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:14.681183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c70cfc-f8ef-4251-90a6-523cc0291c7c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.681617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c70cfc-f8ef-4251-90a6-523cc0291c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.681938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6cb020b5-e1d0-431d-b5d6-e6948462a7b9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:14.682047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c70cfc-f8ef-4251-90a6-523cc0291c7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.683152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6cb020b5-e1d0-431d-b5d6-e6948462a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:14.689998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c70cfc-f8ef-4251-90a6-523cc0291c7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:14.691866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c70cfc-f8ef-4251-90a6-523cc0291c7c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e9c70cfc-f8ef-4251-90a6-523cc0291c7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:14.703890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:14.704785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:14.705699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:14.705699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:14.705699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:14.705699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:14.705699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:14.705699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:14.705699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:14.705699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:14.705699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 313/630] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:14.706189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c70cfc-f8ef-4251-90a6-523cc0291c7c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:14.719772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:14.764058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 318/631] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:14.764858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb020b5-e1d0-431d-b5d6-e6948462a7b9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:14.827883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c70cfc-f8ef-4251-90a6-523cc0291c7c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:14.875053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5840fb-158d-4877-adab-049c0bb801aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:14.875276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5840fb-158d-4877-adab-049c0bb801aa None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.875538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5840fb-158d-4877-adab-049c0bb801aa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.875807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5840fb-158d-4877-adab-049c0bb801aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.889392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb020b5-e1d0-431d-b5d6-e6948462a7b9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:14.898669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:14.926746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660479f-544b-4fbc-8013-da774a759c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:14.927663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660479f-544b-4fbc-8013-da774a759c39 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:14.928864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660479f-544b-4fbc-8013-da774a759c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:14.929285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660479f-544b-4fbc-8013-da774a759c39 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:14.930190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660479f-544b-4fbc-8013-da774a759c39 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:14.930655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660479f-544b-4fbc-8013-da774a759c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:14.932192 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660479f-544b-4fbc-8013-da774a759c39 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:14.948543 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 319/632] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:14.949768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5840fb-158d-4877-adab-049c0bb801aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:14.949978 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660479f-544b-4fbc-8013-da774a759c39 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:14.951346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5840fb-158d-4877-adab-049c0bb801aa None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:14.980877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:15.057851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 314/633] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:15.058711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5840fb-158d-4877-adab-049c0bb801aa None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.059655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660479f-544b-4fbc-8013-da774a759c39 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.096365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b5559-093a-4bc0-8cb8-69918cd98d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.096993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b5559-093a-4bc0-8cb8-69918cd98d36 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.097263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b5559-093a-4bc0-8cb8-69918cd98d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.097529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b5559-093a-4bc0-8cb8-69918cd98d36 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.098203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b5559-093a-4bc0-8cb8-69918cd98d36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:15.098498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b5559-093a-4bc0-8cb8-69918cd98d36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.099922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b5559-093a-4bc0-8cb8-69918cd98d36 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:15.112575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 320/634] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:15.113687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b5559-093a-4bc0-8cb8-69918cd98d36 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.122193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:15.172902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5840fb-158d-4877-adab-049c0bb801aa None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.206780 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:15.209502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b8afa-07d0-4691-aba7-cb2334fcdc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.210138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b8afa-07d0-4691-aba7-cb2334fcdc81 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.210413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b8afa-07d0-4691-aba7-cb2334fcdc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.210681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b8afa-07d0-4691-aba7-cb2334fcdc81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.218228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b5559-093a-4bc0-8cb8-69918cd98d36 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.220827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b8afa-07d0-4691-aba7-cb2334fcdc81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.227220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b8afa-07d0-4691-aba7-cb2334fcdc81 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5e5b8afa-07d0-4691-aba7-cb2334fcdc81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:15.236912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:15.238190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:15.239353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:15.239353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:15.239353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:15.239353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 315/635] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:15.239353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b8afa-07d0-4691-aba7-cb2334fcdc81 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.255221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1e084-764e-441d-be14-505aca68964f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.255844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1e084-764e-441d-be14-505aca68964f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.256117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1e084-764e-441d-be14-505aca68964f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.256381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1e084-764e-441d-be14-505aca68964f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.267071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1e084-764e-441d-be14-505aca68964f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-52e1e084-764e-441d-be14-505aca68964f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:15.272226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:15.273388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:15.274741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:15.278426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 321/636] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:15.279116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1e084-764e-441d-be14-505aca68964f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.343203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b8afa-07d0-4691-aba7-cb2334fcdc81 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.346197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:15.377595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1e084-764e-441d-be14-505aca68964f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.378669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee24c94-fa40-4cd0-a0d6-04dfa939d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.379294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee24c94-fa40-4cd0-a0d6-04dfa939d103 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.379569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee24c94-fa40-4cd0-a0d6-04dfa939d103 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.379845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee24c94-fa40-4cd0-a0d6-04dfa939d103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.390966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee24c94-fa40-4cd0-a0d6-04dfa939d103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.392395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee24c94-fa40-4cd0-a0d6-04dfa939d103 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6ee24c94-fa40-4cd0-a0d6-04dfa939d103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:15.406257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:15.407366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:15.408255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:15.408255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:15.408255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:15.408255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 316/637] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:15.408754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3c03-a584-4088-9619-6fa9dd428400 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.409055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3c03-a584-4088-9619-6fa9dd428400 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.409513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a3c03-a584-4088-9619-6fa9dd428400 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.410811 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 317/638] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:15.413023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571c738-f960-4e6f-b298-5e2d072aa747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.413564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a3c03-a584-4088-9619-6fa9dd428400 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.413682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571c738-f960-4e6f-b298-5e2d072aa747 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.413944 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571c738-f960-4e6f-b298-5e2d072aa747 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.414203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571c738-f960-4e6f-b298-5e2d072aa747 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/1ba54c2312b348b08e0faae745747e7d/roles/69cd1c433d70467a91f4a9f47382a412` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.429441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571c738-f960-4e6f-b298-5e2d072aa747 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=1ba54c2312b348b08e0faae745747e7d, role_id=69cd1c433d70467a91f4a9f47382a412)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.431470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571c738-f960-4e6f-b298-5e2d072aa747 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:15.516707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a3c03-a584-4088-9619-6fa9dd428400 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.536132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:15.541821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 322/639] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:12:15 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/1ba54c2312b348b08e0faae745747e7d/roles/69cd1c433d70467a91f4a9f47382a412 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:15.542622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571c738-f960-4e6f-b298-5e2d072aa747 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.559924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e72935-bbba-457b-adfe-40e546b8e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.560552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e72935-bbba-457b-adfe-40e546b8e470 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.560846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e72935-bbba-457b-adfe-40e546b8e470 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.561251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e72935-bbba-457b-adfe-40e546b8e470 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.561775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e72935-bbba-457b-adfe-40e546b8e470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:15.562189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e72935-bbba-457b-adfe-40e546b8e470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.563615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e72935-bbba-457b-adfe-40e546b8e470 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:15.579172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 318/640] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:15.580152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e72935-bbba-457b-adfe-40e546b8e470 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.605569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:15.664232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571c738-f960-4e6f-b298-5e2d072aa747 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.704304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d54a3-6065-405d-95ab-18b1c637c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.704933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d54a3-6065-405d-95ab-18b1c637c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.705206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d54a3-6065-405d-95ab-18b1c637c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.705467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d54a3-6065-405d-95ab-18b1c637c1f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.706116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d54a3-6065-405d-95ab-18b1c637c1f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:15.706408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d54a3-6065-405d-95ab-18b1c637c1f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.707730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e72935-bbba-457b-adfe-40e546b8e470 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.707835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d54a3-6065-405d-95ab-18b1c637c1f1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:15.719088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 323/641] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:15.720604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88511b8-3fe5-49b6-becd-ea93a64dddd2 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.720956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88511b8-3fe5-49b6-becd-ea93a64dddd2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.721228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88511b8-3fe5-49b6-becd-ea93a64dddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.744458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b88511b8-3fe5-49b6-becd-ea93a64dddd2 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:15.746276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b88511b8-3fe5-49b6-becd-ea93a64dddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:15.748844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93202b37-1320-4c39-b62e-45d6183954e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.749441 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93202b37-1320-4c39-b62e-45d6183954e5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.749714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93202b37-1320-4c39-b62e-45d6183954e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.749981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93202b37-1320-4c39-b62e-45d6183954e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.750618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93202b37-1320-4c39-b62e-45d6183954e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:15.750915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93202b37-1320-4c39-b62e-45d6183954e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.752262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93202b37-1320-4c39-b62e-45d6183954e5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:15.764468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 319/642] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:15.765312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93202b37-1320-4c39-b62e-45d6183954e5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.814995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:15.826444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 324/643] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:15.827490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88511b8-3fe5-49b6-becd-ea93a64dddd2 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.858180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:15.884755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93202b37-1320-4c39-b62e-45d6183954e5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.921250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b67a0-2338-428d-bcc0-51a7942b2492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.921870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b67a0-2338-428d-bcc0-51a7942b2492 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.922197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b67a0-2338-428d-bcc0-51a7942b2492 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.922461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b67a0-2338-428d-bcc0-51a7942b2492 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.931212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b67a0-2338-428d-bcc0-51a7942b2492 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.932966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b67a0-2338-428d-bcc0-51a7942b2492 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:15.943090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88511b8-3fe5-49b6-becd-ea93a64dddd2 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-c78b67a0-2338-428d-bcc0-51a7942b2492 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:15.945266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:15.947002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:15.947976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:15.951894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 320/644] 10.222.0.84 () {56 vars in 1093 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:15.952737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b67a0-2338-428d-bcc0-51a7942b2492 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:15.984748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dfab3-1ae3-44d6-86cc-f3af4e9365b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:15.985756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dfab3-1ae3-44d6-86cc-f3af4e9365b6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:15.986214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dfab3-1ae3-44d6-86cc-f3af4e9365b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:15.986662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979dfab3-1ae3-44d6-86cc-f3af4e9365b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:15.998096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979dfab3-1ae3-44d6-86cc-f3af4e9365b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:15.999794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979dfab3-1ae3-44d6-86cc-f3af4e9365b6 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-979dfab3-1ae3-44d6-86cc-f3af4e9365b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:16.016191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:16.017206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:16.018277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:16.018277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:16.018277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:16.020111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 325/645] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:16.021094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dfab3-1ae3-44d6-86cc-f3af4e9365b6 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.064833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b67a0-2338-428d-bcc0-51a7942b2492 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.065695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:16.100990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ccffb-17df-4b7a-a96d-c2977a4ad818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.101621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ccffb-17df-4b7a-a96d-c2977a4ad818 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.101930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ccffb-17df-4b7a-a96d-c2977a4ad818 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.102442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ccffb-17df-4b7a-a96d-c2977a4ad818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.110771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ccffb-17df-4b7a-a96d-c2977a4ad818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:16.111534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:16.113346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ccffb-17df-4b7a-a96d-c2977a4ad818 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:16.125784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979dfab3-1ae3-44d6-86cc-f3af4e9365b6 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-530ccffb-17df-4b7a-a96d-c2977a4ad818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:16.126986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:16.127837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:16.128691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:16.128691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:16.128691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:16.128691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:16.128691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:16.128691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:16.128691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:16.128691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:16.130371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 321/646] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:16.131376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ccffb-17df-4b7a-a96d-c2977a4ad818 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.167625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11410fd-bb13-4a22-b1e4-987affd5dd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.168349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11410fd-bb13-4a22-b1e4-987affd5dd8a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.168621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11410fd-bb13-4a22-b1e4-987affd5dd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.168926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11410fd-bb13-4a22-b1e4-987affd5dd8a None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.169627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11410fd-bb13-4a22-b1e4-987affd5dd8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:16.169921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11410fd-bb13-4a22-b1e4-987affd5dd8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:16.171329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11410fd-bb13-4a22-b1e4-987affd5dd8a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:16.190583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 326/647] 10.222.0.84 () {56 vars in 1133 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:16.191315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11410fd-bb13-4a22-b1e4-987affd5dd8a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.248683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ccffb-17df-4b7a-a96d-c2977a4ad818 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.284115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e8838-5890-4c0b-bffe-5eab5943efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.284719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e8838-5890-4c0b-bffe-5eab5943efbf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.285022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e8838-5890-4c0b-bffe-5eab5943efbf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.285292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e8838-5890-4c0b-bffe-5eab5943efbf None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.285926 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e8838-5890-4c0b-bffe-5eab5943efbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:16.286228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e8838-5890-4c0b-bffe-5eab5943efbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:16.287618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e8838-5890-4c0b-bffe-5eab5943efbf None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:16.293220 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:16.297166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11410fd-bb13-4a22-b1e4-987affd5dd8a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.301032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 322/648] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:16.306272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e8838-5890-4c0b-bffe-5eab5943efbf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.331594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4c721-a939-4afe-ab73-c26b94628194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.332213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4c721-a939-4afe-ab73-c26b94628194 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.332486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4c721-a939-4afe-ab73-c26b94628194 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.332751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4c721-a939-4afe-ab73-c26b94628194 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.345078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4c721-a939-4afe-ab73-c26b94628194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-56b4c721-a939-4afe-ab73-c26b94628194 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:16.349761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:16.350657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 327/649] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:16.352924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4c721-a939-4afe-ab73-c26b94628194 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.377299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:16.427216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e8838-5890-4c0b-bffe-5eab5943efbf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.465317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66cf94-f8e5-40a9-80e2-45fa46de5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.465924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66cf94-f8e5-40a9-80e2-45fa46de5abe None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.466198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66cf94-f8e5-40a9-80e2-45fa46de5abe None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.466465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66cf94-f8e5-40a9-80e2-45fa46de5abe None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.467146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66cf94-f8e5-40a9-80e2-45fa46de5abe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:16.467450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66cf94-f8e5-40a9-80e2-45fa46de5abe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:16.468891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac66cf94-f8e5-40a9-80e2-45fa46de5abe None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:16.471422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4c721-a939-4afe-ab73-c26b94628194 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.482532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 323/650] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:16.483654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66cf94-f8e5-40a9-80e2-45fa46de5abe None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.513485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40f73-df10-459a-b609-ebb63876521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.514093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40f73-df10-459a-b609-ebb63876521b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.514360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40f73-df10-459a-b609-ebb63876521b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.514612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d40f73-df10-459a-b609-ebb63876521b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.547043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:16.590114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d40f73-df10-459a-b609-ebb63876521b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:16.591590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d40f73-df10-459a-b609-ebb63876521b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:16.594338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66cf94-f8e5-40a9-80e2-45fa46de5abe None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.612733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:16.631516 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f8e13-84a7-4e82-805f-53145a6e74b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.632063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f8e13-84a7-4e82-805f-53145a6e74b4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.632334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f8e13-84a7-4e82-805f-53145a6e74b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.632889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f8e13-84a7-4e82-805f-53145a6e74b4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.640330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f8e13-84a7-4e82-805f-53145a6e74b4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:16.641879 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f8e13-84a7-4e82-805f-53145a6e74b4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d38f8e13-84a7-4e82-805f-53145a6e74b4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:16.655690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:16.656770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 324/651] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:16.657715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f8e13-84a7-4e82-805f-53145a6e74b4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.719881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 328/652] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:16.720765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40f73-df10-459a-b609-ebb63876521b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.763606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f8e13-84a7-4e82-805f-53145a6e74b4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.782268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:16.797423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ebd20-af2b-40e1-b558-a0c1e6ba23b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.798026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ebd20-af2b-40e1-b558-a0c1e6ba23b6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.798304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ebd20-af2b-40e1-b558-a0c1e6ba23b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.798575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ebd20-af2b-40e1-b558-a0c1e6ba23b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.799232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ebd20-af2b-40e1-b558-a0c1e6ba23b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:16.799526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ebd20-af2b-40e1-b558-a0c1e6ba23b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:16.800903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ebd20-af2b-40e1-b558-a0c1e6ba23b6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:16.813087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 325/653] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:16.813087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ebd20-af2b-40e1-b558-a0c1e6ba23b6 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.830777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d40f73-df10-459a-b609-ebb63876521b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.850163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:16.866528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea1143-89f8-407c-8a6e-46bb2c6581ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.867201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea1143-89f8-407c-8a6e-46bb2c6581ff None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.867621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea1143-89f8-407c-8a6e-46bb2c6581ff None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.868024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea1143-89f8-407c-8a6e-46bb2c6581ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.868822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea1143-89f8-407c-8a6e-46bb2c6581ff None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=75ac781cf2d84d95a789724e74e12acf) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:16.869142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea1143-89f8-407c-8a6e-46bb2c6581ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:16.870543 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea1143-89f8-407c-8a6e-46bb2c6581ff None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:16.884910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 329/654] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/role_assignments?user.id=75ac781cf2d84d95a789724e74e12acf&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:16.884910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea1143-89f8-407c-8a6e-46bb2c6581ff None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:16.929871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ebd20-af2b-40e1-b558-a0c1e6ba23b6 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:16.974722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051980a-8093-4078-9c6a-5b99f41dbcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:16.978532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051980a-8093-4078-9c6a-5b99f41dbcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:16.978989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051980a-8093-4078-9c6a-5b99f41dbcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:16.978989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051980a-8093-4078-9c6a-5b99f41dbcb4 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:16.980515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051980a-8093-4078-9c6a-5b99f41dbcb4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:16.980938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051980a-8093-4078-9c6a-5b99f41dbcb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:16.984153 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051980a-8093-4078-9c6a-5b99f41dbcb4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.009958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 326/655] 10.222.0.84 () {56 vars in 1137 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:17.011685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051980a-8093-4078-9c6a-5b99f41dbcb4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:17.029655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea1143-89f8-407c-8a6e-46bb2c6581ff None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:17.064014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:17.076756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8f03b-94c8-44f6-a131-b690d02bb5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:17.077548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8f03b-94c8-44f6-a131-b690d02bb5ef None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.077926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8f03b-94c8-44f6-a131-b690d02bb5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.078299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8f03b-94c8-44f6-a131-b690d02bb5ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.091219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8f03b-94c8-44f6-a131-b690d02bb5ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:17.093858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8f03b-94c8-44f6-a131-b690d02bb5ef None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-29a8f03b-94c8-44f6-a131-b690d02bb5ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:17.111043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:17.112010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:17.112923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:17.112923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:17.112923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:17.114651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 330/656] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:17.119108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8f03b-94c8-44f6-a131-b690d02bb5ef None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:17.141641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:17.150290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051980a-8093-4078-9c6a-5b99f41dbcb4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:17.201902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a60d-7500-4dfb-ae79-529c8a331227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:17.202900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a60d-7500-4dfb-ae79-529c8a331227 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.203378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a60d-7500-4dfb-ae79-529c8a331227 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.203892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59a60d-7500-4dfb-ae79-529c8a331227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.216434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59a60d-7500-4dfb-ae79-529c8a331227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:17.218377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59a60d-7500-4dfb-ae79-529c8a331227 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-1c59a60d-7500-4dfb-ae79-529c8a331227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:17.235465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:17.236437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:17.237333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:17.237333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:17.237333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:17.237518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8f03b-94c8-44f6-a131-b690d02bb5ef None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:17.239663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 327/657] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:17.240654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a60d-7500-4dfb-ae79-529c8a331227 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:17.278682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1eafe-875b-43bd-832e-baabfa2c3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:17.279323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1eafe-875b-43bd-832e-baabfa2c3df0 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.279602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1eafe-875b-43bd-832e-baabfa2c3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.279872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1eafe-875b-43bd-832e-baabfa2c3df0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.343212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:17.361850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59a60d-7500-4dfb-ae79-529c8a331227 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:17.369131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1eafe-875b-43bd-832e-baabfa2c3df0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:17.370939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1eafe-875b-43bd-832e-baabfa2c3df0 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.408996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ea19a-f3c8-4cfd-a86e-7118bc243513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:17.409529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ea19a-f3c8-4cfd-a86e-7118bc243513 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.409782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ea19a-f3c8-4cfd-a86e-7118bc243513 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.410053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ea19a-f3c8-4cfd-a86e-7118bc243513 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.410690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ea19a-f3c8-4cfd-a86e-7118bc243513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:17.410990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ea19a-f3c8-4cfd-a86e-7118bc243513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:17.412346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ea19a-f3c8-4cfd-a86e-7118bc243513 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.431174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 328/658] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:17.432464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ea19a-f3c8-4cfd-a86e-7118bc243513 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:17.444675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:17.497348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 331/659] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:17.498214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1eafe-875b-43bd-832e-baabfa2c3df0 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:17.547605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ea19a-f3c8-4cfd-a86e-7118bc243513 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:17.583736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:17.591383 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebde05-e3e9-4863-955a-07d746b6d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:17.592168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebde05-e3e9-4863-955a-07d746b6d087 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.592729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebde05-e3e9-4863-955a-07d746b6d087 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.593039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebde05-e3e9-4863-955a-07d746b6d087 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.593690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebde05-e3e9-4863-955a-07d746b6d087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:17.597138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebde05-e3e9-4863-955a-07d746b6d087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:17.597138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebde05-e3e9-4863-955a-07d746b6d087 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.607712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1eafe-875b-43bd-832e-baabfa2c3df0 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:17.611852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 329/660] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:17.612844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebde05-e3e9-4863-955a-07d746b6d087 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:17.643609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91048a6-7149-4fd2-b9f5-032931e33929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:17.644233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91048a6-7149-4fd2-b9f5-032931e33929 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.644506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91048a6-7149-4fd2-b9f5-032931e33929 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.644772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91048a6-7149-4fd2-b9f5-032931e33929 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.656753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91048a6-7149-4fd2-b9f5-032931e33929 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:17.658269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91048a6-7149-4fd2-b9f5-032931e33929 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-f91048a6-7149-4fd2-b9f5-032931e33929 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:17.670641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:17.671700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 332/661] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:17.672904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b642de-db9b-49a9-9fcd-138111f93e7a None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.673502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b642de-db9b-49a9-9fcd-138111f93e7a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.673502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b642de-db9b-49a9-9fcd-138111f93e7a None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.674727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 333/662] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:17.675231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b642de-db9b-49a9-9fcd-138111f93e7a None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:17.699188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:17.735412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebde05-e3e9-4863-955a-07d746b6d087 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:17.769541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb62c5b-0376-4718-bd24-f315c5851c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:17.770316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb62c5b-0376-4718-bd24-f315c5851c8a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.770687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb62c5b-0376-4718-bd24-f315c5851c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.771076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb62c5b-0376-4718-bd24-f315c5851c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.782684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb62c5b-0376-4718-bd24-f315c5851c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:17.784167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb62c5b-0376-4718-bd24-f315c5851c8a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.791343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b642de-db9b-49a9-9fcd-138111f93e7a None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5fb62c5b-0376-4718-bd24-f315c5851c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:17.798558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:17.799641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:17.805413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:17.805413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:17.805413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:17.805413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 330/663] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:17.805413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb62c5b-0376-4718-bd24-f315c5851c8a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:17.833084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ed03f-93a9-4c1a-85d5-7f918cfeb91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:17.833788 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ed03f-93a9-4c1a-85d5-7f918cfeb91a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.834134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ed03f-93a9-4c1a-85d5-7f918cfeb91a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.834567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ed03f-93a9-4c1a-85d5-7f918cfeb91a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.838449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:17.843024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ed03f-93a9-4c1a-85d5-7f918cfeb91a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:17.844462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ed03f-93a9-4c1a-85d5-7f918cfeb91a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-cc2ed03f-93a9-4c1a-85d5-7f918cfeb91a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:17.857882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:17.858857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:17.859789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 334/664] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:17.860522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436dad7-9a89-4f6d-9e87-9cf1deb5e22c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.860522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436dad7-9a89-4f6d-9e87-9cf1deb5e22c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.860652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436dad7-9a89-4f6d-9e87-9cf1deb5e22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.879357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-1436dad7-9a89-4f6d-9e87-9cf1deb5e22c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:17.880553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1436dad7-9a89-4f6d-9e87-9cf1deb5e22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:17.918018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb62c5b-0376-4718-bd24-f315c5851c8a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:17.941210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:17.960824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 335/665] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:17.960824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436dad7-9a89-4f6d-9e87-9cf1deb5e22c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:17.965274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee96c7-f2fd-409d-bb5c-0cdf5d8eeee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:17.966001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee96c7-f2fd-409d-bb5c-0cdf5d8eeee1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:17.966349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee96c7-f2fd-409d-bb5c-0cdf5d8eeee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:17.966786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee96c7-f2fd-409d-bb5c-0cdf5d8eeee1 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:17.967544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee96c7-f2fd-409d-bb5c-0cdf5d8eeee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:17.967941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee96c7-f2fd-409d-bb5c-0cdf5d8eeee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:17.969621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee96c7-f2fd-409d-bb5c-0cdf5d8eeee1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:17.986678 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 331/666] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:17.987720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee96c7-f2fd-409d-bb5c-0cdf5d8eeee1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.069252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436dad7-9a89-4f6d-9e87-9cf1deb5e22c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.103048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:18.115324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c2ce0-3ab3-458c-bd5a-6b9b4a6d5f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.116129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c2ce0-3ab3-458c-bd5a-6b9b4a6d5f43 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.116476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee96c7-f2fd-409d-bb5c-0cdf5d8eeee1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.116599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c2ce0-3ab3-458c-bd5a-6b9b4a6d5f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.116964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c2ce0-3ab3-458c-bd5a-6b9b4a6d5f43 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.118289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c2ce0-3ab3-458c-bd5a-6b9b4a6d5f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:18.118709 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c2ce0-3ab3-458c-bd5a-6b9b4a6d5f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.120994 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c2ce0-3ab3-458c-bd5a-6b9b4a6d5f43 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.138261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 336/667] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:18.141677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c2ce0-3ab3-458c-bd5a-6b9b4a6d5f43 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.164206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51608eb-3dff-4780-a7ab-cddf55a0dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.165245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51608eb-3dff-4780-a7ab-cddf55a0dce1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.165620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51608eb-3dff-4780-a7ab-cddf55a0dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.165984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51608eb-3dff-4780-a7ab-cddf55a0dce1 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.167267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51608eb-3dff-4780-a7ab-cddf55a0dce1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:18.167668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51608eb-3dff-4780-a7ab-cddf55a0dce1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.169454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51608eb-3dff-4780-a7ab-cddf55a0dce1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.193059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 332/668] 10.222.0.84 () {56 vars in 1139 bytes} [Wed Aug 24 11:12:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:18.194078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51608eb-3dff-4780-a7ab-cddf55a0dce1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.246802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:18.256751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c2ce0-3ab3-458c-bd5a-6b9b4a6d5f43 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.293110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246b360-2fc1-4fd0-aff0-b55005eaf907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.293878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246b360-2fc1-4fd0-aff0-b55005eaf907 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.294261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246b360-2fc1-4fd0-aff0-b55005eaf907 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.294616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246b360-2fc1-4fd0-aff0-b55005eaf907 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.295597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246b360-2fc1-4fd0-aff0-b55005eaf907 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=f58a559951c94d68951da78c82341a00) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:18.295997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246b360-2fc1-4fd0-aff0-b55005eaf907 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.297611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246b360-2fc1-4fd0-aff0-b55005eaf907 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.312841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 337/669] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:18 2022] GET /identity/v3/role_assignments?user.id=f58a559951c94d68951da78c82341a00&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:18.312841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246b360-2fc1-4fd0-aff0-b55005eaf907 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.324152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51608eb-3dff-4780-a7ab-cddf55a0dce1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.363499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f4380-a7ec-4171-bf3d-0c384016f2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.364421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f4380-a7ec-4171-bf3d-0c384016f2df None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.364760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f4380-a7ec-4171-bf3d-0c384016f2df None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.365123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f4380-a7ec-4171-bf3d-0c384016f2df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.374058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f4380-a7ec-4171-bf3d-0c384016f2df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.375649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f4380-a7ec-4171-bf3d-0c384016f2df None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.376999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-1e7f4380-a7ec-4171-bf3d-0c384016f2df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:18.390896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:18.391858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:18.392760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:18.392760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:18.392760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:18.392760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:18.392760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:18.392760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:18.392760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:18.392760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:18.397214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 333/670] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:18.398176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f4380-a7ec-4171-bf3d-0c384016f2df None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.481104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246b360-2fc1-4fd0-aff0-b55005eaf907 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.518027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712f462-f8da-44fa-acc8-c1292085099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.520737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712f462-f8da-44fa-acc8-c1292085099f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.521858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712f462-f8da-44fa-acc8-c1292085099f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.524939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712f462-f8da-44fa-acc8-c1292085099f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.540866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:18.546914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f4380-a7ec-4171-bf3d-0c384016f2df None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.594750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12b25b-c0f9-4738-ac7c-26e5e07c2eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.595784 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12b25b-c0f9-4738-ac7c-26e5e07c2eeb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.596068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12b25b-c0f9-4738-ac7c-26e5e07c2eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.596346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12b25b-c0f9-4738-ac7c-26e5e07c2eeb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.613285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12b25b-c0f9-4738-ac7c-26e5e07c2eeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-de12b25b-c0f9-4738-ac7c-26e5e07c2eeb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:18.618906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:18.620083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 334/671] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:18.621345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12b25b-c0f9-4738-ac7c-26e5e07c2eeb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.635983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712f462-f8da-44fa-acc8-c1292085099f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.637631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712f462-f8da-44fa-acc8-c1292085099f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.704787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:18.737681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12b25b-c0f9-4738-ac7c-26e5e07c2eeb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.770528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 338/672] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:18.771495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712f462-f8da-44fa-acc8-c1292085099f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.772906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3414cc-12d5-49f2-b94d-09c86d39eeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.773517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3414cc-12d5-49f2-b94d-09c86d39eeb2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.773880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3414cc-12d5-49f2-b94d-09c86d39eeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.774149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3414cc-12d5-49f2-b94d-09c86d39eeb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.774845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3414cc-12d5-49f2-b94d-09c86d39eeb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:18.775148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3414cc-12d5-49f2-b94d-09c86d39eeb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.776580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3414cc-12d5-49f2-b94d-09c86d39eeb2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.789124 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 335/673] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:18.790632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3414cc-12d5-49f2-b94d-09c86d39eeb2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.846245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:18.907163 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712f462-f8da-44fa-acc8-c1292085099f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.923655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3414cc-12d5-49f2-b94d-09c86d39eeb2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:18.946648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3ef13-88f9-4d1e-a72d-70de21a4e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.947532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3ef13-88f9-4d1e-a72d-70de21a4e022 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.948067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3ef13-88f9-4d1e-a72d-70de21a4e022 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.948493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3ef13-88f9-4d1e-a72d-70de21a4e022 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.949350 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3ef13-88f9-4d1e-a72d-70de21a4e022 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:18.949798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3ef13-88f9-4d1e-a72d-70de21a4e022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.951422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3ef13-88f9-4d1e-a72d-70de21a4e022 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.960656 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 339/674] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:18.962355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3ef13-88f9-4d1e-a72d-70de21a4e022 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:18.964251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8b823-f352-4472-83c3-d4286e875053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:18.964914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8b823-f352-4472-83c3-d4286e875053 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:18.965252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8b823-f352-4472-83c3-d4286e875053 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:18.965525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8b823-f352-4472-83c3-d4286e875053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:18.979657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:18.980018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8b823-f352-4472-83c3-d4286e875053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:18.982683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8b823-f352-4472-83c3-d4286e875053 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-30b8b823-f352-4472-83c3-d4286e875053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:18.999862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:19.000914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:19.001807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:19.001807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:19.001807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:19.001807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 336/675] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:19.001807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8b823-f352-4472-83c3-d4286e875053 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.072429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:19.072636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3ef13-88f9-4d1e-a72d-70de21a4e022 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.105995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8b823-f352-4472-83c3-d4286e875053 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.109410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976ce35-f0b5-4e00-8ea7-6a968bb8986f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.110016 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976ce35-f0b5-4e00-8ea7-6a968bb8986f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.110294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976ce35-f0b5-4e00-8ea7-6a968bb8986f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.110564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976ce35-f0b5-4e00-8ea7-6a968bb8986f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.111280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976ce35-f0b5-4e00-8ea7-6a968bb8986f None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=d5ce16d5ea8240debfcb69ac7b1f71c0) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:19.111583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976ce35-f0b5-4e00-8ea7-6a968bb8986f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.113036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976ce35-f0b5-4e00-8ea7-6a968bb8986f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.125048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 340/676] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:18 2022] GET /identity/v3/role_assignments?user.id=d5ce16d5ea8240debfcb69ac7b1f71c0&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:19.127609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21367e51-d2d1-43f0-b195-668395842b44 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.128178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21367e51-d2d1-43f0-b195-668395842b44 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.128870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21367e51-d2d1-43f0-b195-668395842b44 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.135814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 341/677] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:19.135814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21367e51-d2d1-43f0-b195-668395842b44 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.145669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d2de8-d771-4659-affe-cffe86b8a50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.146319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d2de8-d771-4659-affe-cffe86b8a50a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.146592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d2de8-d771-4659-affe-cffe86b8a50a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.146862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d2de8-d771-4659-affe-cffe86b8a50a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.160132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d2de8-d771-4659-affe-cffe86b8a50a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.161300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d2de8-d771-4659-affe-cffe86b8a50a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-848d2de8-d771-4659-affe-cffe86b8a50a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:19.178189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:19.179233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:19.180138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:19.180138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:19.180138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:19.180138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 337/678] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:19.180978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b944-0416-4ad0-96ee-64aae577ed18 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.181097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b944-0416-4ad0-96ee-64aae577ed18 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.181361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016b944-0416-4ad0-96ee-64aae577ed18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.201070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c016b944-0416-4ad0-96ee-64aae577ed18 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:19.202395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c016b944-0416-4ad0-96ee-64aae577ed18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:19.251991 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:19.277762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 338/679] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:19.278362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016b944-0416-4ad0-96ee-64aae577ed18 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.293833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21367e51-d2d1-43f0-b195-668395842b44 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.337709 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee2f02-743d-4cbc-9dc5-73ea9caaeb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.338370 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee2f02-743d-4cbc-9dc5-73ea9caaeb0a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.338647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee2f02-743d-4cbc-9dc5-73ea9caaeb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.338915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee2f02-743d-4cbc-9dc5-73ea9caaeb0a None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.339803 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee2f02-743d-4cbc-9dc5-73ea9caaeb0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:19.340137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee2f02-743d-4cbc-9dc5-73ea9caaeb0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.341776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee2f02-743d-4cbc-9dc5-73ea9caaeb0a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.343751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:19.359229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 342/680] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:19.365324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee2f02-743d-4cbc-9dc5-73ea9caaeb0a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.402944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016b944-0416-4ad0-96ee-64aae577ed18 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.443187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d2fea-fc91-4b0e-9742-dcdb5fbcebda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.443887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d2fea-fc91-4b0e-9742-dcdb5fbcebda None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.444165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d2fea-fc91-4b0e-9742-dcdb5fbcebda None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.444434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d2fea-fc91-4b0e-9742-dcdb5fbcebda None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.445110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d2fea-fc91-4b0e-9742-dcdb5fbcebda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:19.445406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d2fea-fc91-4b0e-9742-dcdb5fbcebda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.447497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d2fea-fc91-4b0e-9742-dcdb5fbcebda None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.463500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 339/681] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:19.466826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d2fea-fc91-4b0e-9742-dcdb5fbcebda None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.471480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee2f02-743d-4cbc-9dc5-73ea9caaeb0a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.510277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1211d6-1308-430d-a23a-1a2d7f49a705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.510823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1211d6-1308-430d-a23a-1a2d7f49a705 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.511097 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1211d6-1308-430d-a23a-1a2d7f49a705 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.511360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1211d6-1308-430d-a23a-1a2d7f49a705 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.514271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:19.518839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1211d6-1308-430d-a23a-1a2d7f49a705 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.520329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1211d6-1308-430d-a23a-1a2d7f49a705 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-4f1211d6-1308-430d-a23a-1a2d7f49a705 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:19.541439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:19.542437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:19.543559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:19.543559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:19.543559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:19.543559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:19.543559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:19.543559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:19.543559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:19.543559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:19.543559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 343/682] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:19.548413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1211d6-1308-430d-a23a-1a2d7f49a705 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.586006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d2fea-fc91-4b0e-9742-dcdb5fbcebda None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.613226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:19.630801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff5df4-95af-486a-95eb-104f0289aeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.631958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff5df4-95af-486a-95eb-104f0289aeaf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.634070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff5df4-95af-486a-95eb-104f0289aeaf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.634507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff5df4-95af-486a-95eb-104f0289aeaf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.653636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff5df4-95af-486a-95eb-104f0289aeaf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.655493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff5df4-95af-486a-95eb-104f0289aeaf None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-afff5df4-95af-486a-95eb-104f0289aeaf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:19.683200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:19.684288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:19.685620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:19.691960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1211d6-1308-430d-a23a-1a2d7f49a705 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.692263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 340/683] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:19.693321 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff5df4-95af-486a-95eb-104f0289aeaf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.737148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1040a-d527-41e7-8a4f-c99924a0ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.737768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1040a-d527-41e7-8a4f-c99924a0ed86 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.738048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1040a-d527-41e7-8a4f-c99924a0ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.738318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1040a-d527-41e7-8a4f-c99924a0ed86 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.739009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1040a-d527-41e7-8a4f-c99924a0ed86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:19.739310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1040a-d527-41e7-8a4f-c99924a0ed86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.741361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1040a-d527-41e7-8a4f-c99924a0ed86 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.757316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 344/684] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:19.759257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1040a-d527-41e7-8a4f-c99924a0ed86 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.800950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff5df4-95af-486a-95eb-104f0289aeaf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.834287 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:19.843871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02b925-4340-48e3-9c71-dbc72fc08c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.848743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02b925-4340-48e3-9c71-dbc72fc08c3d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.848743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02b925-4340-48e3-9c71-dbc72fc08c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.848743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02b925-4340-48e3-9c71-dbc72fc08c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.860433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02b925-4340-48e3-9c71-dbc72fc08c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.863032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02b925-4340-48e3-9c71-dbc72fc08c3d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-8d02b925-4340-48e3-9c71-dbc72fc08c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:19.882581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:19.884639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:19.885577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:19.885577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:19.885577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:19.885766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1040a-d527-41e7-8a4f-c99924a0ed86 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:19.888862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 341/685] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:19.889902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02b925-4340-48e3-9c71-dbc72fc08c3d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:19.922829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:19.932001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18939019-5556-4b51-960f-a14d4bf9fd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:19.934605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18939019-5556-4b51-960f-a14d4bf9fd9b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:19.934605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18939019-5556-4b51-960f-a14d4bf9fd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:19.934605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18939019-5556-4b51-960f-a14d4bf9fd9b None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:19.934605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18939019-5556-4b51-960f-a14d4bf9fd9b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:19.934605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18939019-5556-4b51-960f-a14d4bf9fd9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:19.936637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18939019-5556-4b51-960f-a14d4bf9fd9b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:19.958146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 345/686] 10.222.0.84 () {56 vars in 1135 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:19.958999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18939019-5556-4b51-960f-a14d4bf9fd9b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:20.018635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02b925-4340-48e3-9c71-dbc72fc08c3d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:20.054633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a86bf-fcac-431a-b33b-8ff4a8d0bd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:20.055801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a86bf-fcac-431a-b33b-8ff4a8d0bd54 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:20.056043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a86bf-fcac-431a-b33b-8ff4a8d0bd54 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:20.056369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a86bf-fcac-431a-b33b-8ff4a8d0bd54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:20.057137 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a86bf-fcac-431a-b33b-8ff4a8d0bd54 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=372e96bd3361434e9169b783f0c9ea13) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:20.057443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a86bf-fcac-431a-b33b-8ff4a8d0bd54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:20.058872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a86bf-fcac-431a-b33b-8ff4a8d0bd54 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:20.065158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 342/687] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/role_assignments?user.id=372e96bd3361434e9169b783f0c9ea13&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:20.066012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a86bf-fcac-431a-b33b-8ff4a8d0bd54 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:20.103127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18939019-5556-4b51-960f-a14d4bf9fd9b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:20.124405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:20.153215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90819932-005f-4719-9439-f5dffed52757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:20.154291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90819932-005f-4719-9439-f5dffed52757 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:20.154575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90819932-005f-4719-9439-f5dffed52757 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:20.154848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90819932-005f-4719-9439-f5dffed52757 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:20.156171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90819932-005f-4719-9439-f5dffed52757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:20.156976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90819932-005f-4719-9439-f5dffed52757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:20.158843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90819932-005f-4719-9439-f5dffed52757 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:20.167912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:20.178994 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 346/688] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:20.185605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90819932-005f-4719-9439-f5dffed52757 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:20.186703 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a86bf-fcac-431a-b33b-8ff4a8d0bd54 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:20.237071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b44ce-7df4-4a7f-aeac-69ac5dc9c407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:20.237696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b44ce-7df4-4a7f-aeac-69ac5dc9c407 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:20.238450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b44ce-7df4-4a7f-aeac-69ac5dc9c407 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:20.238735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b44ce-7df4-4a7f-aeac-69ac5dc9c407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:20.311924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90819932-005f-4719-9439-f5dffed52757 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:20.341939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b44ce-7df4-4a7f-aeac-69ac5dc9c407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:20.343540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b44ce-7df4-4a7f-aeac-69ac5dc9c407 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:20.371315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f947f80-ab75-49b0-9cfc-8f7d69552eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:20.371927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f947f80-ab75-49b0-9cfc-8f7d69552eac None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:20.372207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f947f80-ab75-49b0-9cfc-8f7d69552eac None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:20.372585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f947f80-ab75-49b0-9cfc-8f7d69552eac None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:20.373393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f947f80-ab75-49b0-9cfc-8f7d69552eac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:20.374839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f947f80-ab75-49b0-9cfc-8f7d69552eac None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:20.451208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:20.463384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 347/689] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:12:20 2022] POST /identity/v3/users => generated 254 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:20.469794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 343/690] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:20.475861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f947f80-ab75-49b0-9cfc-8f7d69552eac None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:20.515811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:20.597256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f947f80-ab75-49b0-9cfc-8f7d69552eac None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:20.634216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930bd71-ed69-49ae-84f4-459b829744fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:20.635238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930bd71-ed69-49ae-84f4-459b829744fa None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:20.635711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930bd71-ed69-49ae-84f4-459b829744fa None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:20.636175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930bd71-ed69-49ae-84f4-459b829744fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:20.647677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930bd71-ed69-49ae-84f4-459b829744fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:20.649652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930bd71-ed69-49ae-84f4-459b829744fa None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-7930bd71-ed69-49ae-84f4-459b829744fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:20.664929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:20.666049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:20.666926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:20.666926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:20.666926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:20.670682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 348/691] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:20.672077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b44ce-7df4-4a7f-aeac-69ac5dc9c407 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:20.744351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1ab36-4d7c-4f76-8b52-cbab3187ca89 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:20.745036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1ab36-4d7c-4f76-8b52-cbab3187ca89 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:20.745518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1ab36-4d7c-4f76-8b52-cbab3187ca89 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:20.747204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 349/692] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:12:20.752524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14bd52-b943-4993-b5f2-604ab6ab4f89 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:20.753022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14bd52-b943-4993-b5f2-604ab6ab4f89 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:20.753394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14bd52-b943-4993-b5f2-604ab6ab4f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:20.774496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9a14bd52-b943-4993-b5f2-604ab6ab4f89 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:20.775844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9a14bd52-b943-4993-b5f2-604ab6ab4f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:20.784504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b44ce-7df4-4a7f-aeac-69ac5dc9c407 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:20.836293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b0d09-f5c4-4a37-babf-992f5db05353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:20.837260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b0d09-f5c4-4a37-babf-992f5db05353 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:20.837710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b0d09-f5c4-4a37-babf-992f5db05353 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:20.838115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b0d09-f5c4-4a37-babf-992f5db05353 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:20.839433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b0d09-f5c4-4a37-babf-992f5db05353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:20.840149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b0d09-f5c4-4a37-babf-992f5db05353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:20.841955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b0d09-f5c4-4a37-babf-992f5db05353 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:20.862591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 344/693] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:20.865406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b0d09-f5c4-4a37-babf-992f5db05353 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:20.869735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 350/694] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:12:20.941275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:20.976136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b0d09-f5c4-4a37-babf-992f5db05353 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:21.014340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ccc01-e3eb-4b76-b0c6-091803d3b2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:21.015225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ccc01-e3eb-4b76-b0c6-091803d3b2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:21.015642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ccc01-e3eb-4b76-b0c6-091803d3b2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:21.016139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ccc01-e3eb-4b76-b0c6-091803d3b2c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:21.023219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ccc01-e3eb-4b76-b0c6-091803d3b2c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:21.025373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ccc01-e3eb-4b76-b0c6-091803d3b2c2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6c7ccc01-e3eb-4b76-b0c6-091803d3b2c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:21.037197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:21.038272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:21.039933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:21.039933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:21.039933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:21.039933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:21.039933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:21.039933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:21.039933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:21.039933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:21.046496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 345/695] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:21.048007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14bd52-b943-4993-b5f2-604ab6ab4f89 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:21.063885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:21.154173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14bd52-b943-4993-b5f2-604ab6ab4f89 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:21.189521 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5fc9c-714c-4992-8f65-ca270a045883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:21.190371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5fc9c-714c-4992-8f65-ca270a045883 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:21.190790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5fc9c-714c-4992-8f65-ca270a045883 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:21.191229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5fc9c-714c-4992-8f65-ca270a045883 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:21.192106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5fc9c-714c-4992-8f65-ca270a045883 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:21.192545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5fc9c-714c-4992-8f65-ca270a045883 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:21.194128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5fc9c-714c-4992-8f65-ca270a045883 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:21.211720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 351/696] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:21.213418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ccc01-e3eb-4b76-b0c6-091803d3b2c2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:21.224083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5fc9c-714c-4992-8f65-ca270a045883 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:21.338476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ccc01-e3eb-4b76-b0c6-091803d3b2c2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:21.355016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:21.377056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e20b3-bd50-4aa4-a716-c4520025247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:21.377683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e20b3-bd50-4aa4-a716-c4520025247d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:21.381167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e20b3-bd50-4aa4-a716-c4520025247d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:21.381167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e20b3-bd50-4aa4-a716-c4520025247d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:21.391433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e20b3-bd50-4aa4-a716-c4520025247d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-452e20b3-bd50-4aa4-a716-c4520025247d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:21.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:21.397501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:21.398314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 346/697] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:21.409869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e20b3-bd50-4aa4-a716-c4520025247d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:21.413003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:21.445572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5fc9c-714c-4992-8f65-ca270a045883 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:21.486836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b37c49-bb0a-4f46-8958-3203107afba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:21.487494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b37c49-bb0a-4f46-8958-3203107afba8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:21.487719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b37c49-bb0a-4f46-8958-3203107afba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:21.487914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b37c49-bb0a-4f46-8958-3203107afba8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:21.488755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b37c49-bb0a-4f46-8958-3203107afba8 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=1ba54c2312b348b08e0faae745747e7d) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:21.489105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b37c49-bb0a-4f46-8958-3203107afba8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:21.490545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b37c49-bb0a-4f46-8958-3203107afba8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:21.497472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 352/698] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:21 2022] GET /identity/v3/role_assignments?user.id=1ba54c2312b348b08e0faae745747e7d&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:21.498903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5080779-0cef-47c5-9cc2-555e1455913e None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:21.512934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5080779-0cef-47c5-9cc2-555e1455913e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:21.512934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5080779-0cef-47c5-9cc2-555e1455913e None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:21.512934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 353/699] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:21.512934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2538-37a4-4616-b66d-96947b71b77f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:21.512934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2538-37a4-4616-b66d-96947b71b77f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:21.512934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2538-37a4-4616-b66d-96947b71b77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:21.536760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e20b3-bd50-4aa4-a716-c4520025247d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:21.571020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-5a2b2538-37a4-4616-b66d-96947b71b77f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:21.572955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5a2b2538-37a4-4616-b66d-96947b71b77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:21.577433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aea464-d09b-4731-92e2-10fff906b377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:21.578050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aea464-d09b-4731-92e2-10fff906b377 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:21.578324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aea464-d09b-4731-92e2-10fff906b377 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:21.578593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aea464-d09b-4731-92e2-10fff906b377 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:21.579266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aea464-d09b-4731-92e2-10fff906b377 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:21.579568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aea464-d09b-4731-92e2-10fff906b377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:21.580979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aea464-d09b-4731-92e2-10fff906b377 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:21.592131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 347/700] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:21.597565 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aea464-d09b-4731-92e2-10fff906b377 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:21.638415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:21.707082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 354/701] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:21.757020 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aea464-d09b-4731-92e2-10fff906b377 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:21.792616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133f9e2-9dde-4172-88f3-f8fcd4165247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:21.793539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133f9e2-9dde-4172-88f3-f8fcd4165247 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:21.793967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133f9e2-9dde-4172-88f3-f8fcd4165247 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:21.794399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133f9e2-9dde-4172-88f3-f8fcd4165247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:21.806116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133f9e2-9dde-4172-88f3-f8fcd4165247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:21.807756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133f9e2-9dde-4172-88f3-f8fcd4165247 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6133f9e2-9dde-4172-88f3-f8fcd4165247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:21.822446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:21.823601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:21.824705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:21.824705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:21.824705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:21.826389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 348/702] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:21.832205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b2538-37a4-4616-b66d-96947b71b77f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:21.904249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:21.940637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b2538-37a4-4616-b66d-96947b71b77f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:21.992719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8aaee0-bb56-4714-a6b5-a3190f53cbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:21.993649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8aaee0-bb56-4714-a6b5-a3190f53cbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:21.994098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8aaee0-bb56-4714-a6b5-a3190f53cbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:21.994530 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8aaee0-bb56-4714-a6b5-a3190f53cbb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:21.995325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8aaee0-bb56-4714-a6b5-a3190f53cbb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:21.995782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8aaee0-bb56-4714-a6b5-a3190f53cbb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:21.997475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8aaee0-bb56-4714-a6b5-a3190f53cbb3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:22.015503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133f9e2-9dde-4172-88f3-f8fcd4165247 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:22.015812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 355/703] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:22.020013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8aaee0-bb56-4714-a6b5-a3190f53cbb3 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:22.106958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:22.130392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8aaee0-bb56-4714-a6b5-a3190f53cbb3 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:22.130603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133f9e2-9dde-4172-88f3-f8fcd4165247 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:22.163242 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:22.171239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab0589-eb08-4bd1-a997-22da01d7ed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:22.172110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab0589-eb08-4bd1-a997-22da01d7ed76 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:22.172538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab0589-eb08-4bd1-a997-22da01d7ed76 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:22.172999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab0589-eb08-4bd1-a997-22da01d7ed76 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:22.176268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26caa6f2-fad9-4c50-9c36-728312b2a7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:22.176411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab0589-eb08-4bd1-a997-22da01d7ed76 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:22.176506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caa6f2-fad9-4c50-9c36-728312b2a7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:22.176506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caa6f2-fad9-4c50-9c36-728312b2a7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:22.176506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caa6f2-fad9-4c50-9c36-728312b2a7f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:22.176716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab0589-eb08-4bd1-a997-22da01d7ed76 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:22.183855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26caa6f2-fad9-4c50-9c36-728312b2a7f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:22.185345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26caa6f2-fad9-4c50-9c36-728312b2a7f1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-65ab0589-eb08-4bd1-a997-22da01d7ed76 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:22.187892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:22.189198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:12:22.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:22.193126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 349/704] 10.222.0.84 () {56 vars in 1129 bytes} [Wed Aug 24 11:12:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-26caa6f2-fad9-4c50-9c36-728312b2a7f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:22.198714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:22.199690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:22.200625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:22.201315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab0589-eb08-4bd1-a997-22da01d7ed76 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:22.202774 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 356/705] 10.222.0.84 () {56 vars in 1093 bytes} [Wed Aug 24 11:12:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:22.217857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26caa6f2-fad9-4c50-9c36-728312b2a7f1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:22.318975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab0589-eb08-4bd1-a997-22da01d7ed76 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:22.358216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aadf75-5c11-4d63-bf92-95cacadf5d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:22.359287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aadf75-5c11-4d63-bf92-95cacadf5d4a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:22.359634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aadf75-5c11-4d63-bf92-95cacadf5d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:22.359907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aadf75-5c11-4d63-bf92-95cacadf5d4a None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:22.360547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aadf75-5c11-4d63-bf92-95cacadf5d4a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:22.360932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aadf75-5c11-4d63-bf92-95cacadf5d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:22.362748 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aadf75-5c11-4d63-bf92-95cacadf5d4a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:22.372830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 350/706] 10.222.0.84 () {56 vars in 1124 bytes} [Wed Aug 24 11:12:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:22.375030 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aadf75-5c11-4d63-bf92-95cacadf5d4a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:22.389160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:22.490215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aadf75-5c11-4d63-bf92-95cacadf5d4a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:22.515619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26caa6f2-fad9-4c50-9c36-728312b2a7f1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:22.526142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6616f4-12d7-4b3e-9bd6-54ffbb8467b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:22.526755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6616f4-12d7-4b3e-9bd6-54ffbb8467b5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:22.527032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6616f4-12d7-4b3e-9bd6-54ffbb8467b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:22.527310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6616f4-12d7-4b3e-9bd6-54ffbb8467b5 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:22.527938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6616f4-12d7-4b3e-9bd6-54ffbb8467b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:22.528238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6616f4-12d7-4b3e-9bd6-54ffbb8467b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:22.529691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6616f4-12d7-4b3e-9bd6-54ffbb8467b5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:22.542840 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 351/707] 10.222.0.84 () {56 vars in 1124 bytes} [Wed Aug 24 11:12:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:22.564535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8fd42-22ba-4c1c-80b7-e98cb6331ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:22.565171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8fd42-22ba-4c1c-80b7-e98cb6331ecd None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:22.565443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8fd42-22ba-4c1c-80b7-e98cb6331ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:22.565707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8fd42-22ba-4c1c-80b7-e98cb6331ecd None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:22.566426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8fd42-22ba-4c1c-80b7-e98cb6331ecd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:22.571107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8fd42-22ba-4c1c-80b7-e98cb6331ecd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:22.571107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8fd42-22ba-4c1c-80b7-e98cb6331ecd None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:22.582928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8292c6b-5712-4724-8ded-d568c6e8b214 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:22.583205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8292c6b-5712-4724-8ded-d568c6e8b214 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:22.583630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8292c6b-5712-4724-8ded-d568c6e8b214 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:22.585504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 352/708] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:22.591750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e073f-f00d-46ac-88b8-f96c2e82bfe8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:22.592443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e073f-f00d-46ac-88b8-f96c2e82bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:22.592964 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e073f-f00d-46ac-88b8-f96c2e82bfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:22.601663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 357/709] 10.222.0.84 () {56 vars in 1133 bytes} [Wed Aug 24 11:12:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:22.609178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8fd42-22ba-4c1c-80b7-e98cb6331ecd None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:22.637946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-371e073f-f00d-46ac-88b8-f96c2e82bfe8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:22.640669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-371e073f-f00d-46ac-88b8-f96c2e82bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:22.704338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:22.770399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 353/710] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:22.792345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8fd42-22ba-4c1c-80b7-e98cb6331ecd None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:22.829235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3480908-2726-49d0-b3de-b795f3e1e33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:22.829756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3480908-2726-49d0-b3de-b795f3e1e33f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:22.830029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3480908-2726-49d0-b3de-b795f3e1e33f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:22.830292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3480908-2726-49d0-b3de-b795f3e1e33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:22.900734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3480908-2726-49d0-b3de-b795f3e1e33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:22.902221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3480908-2726-49d0-b3de-b795f3e1e33f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:22.947525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:23.003602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 358/711] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:23.009667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e073f-f00d-46ac-88b8-f96c2e82bfe8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:23.018814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:23.169050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e073f-f00d-46ac-88b8-f96c2e82bfe8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:23.169221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3480908-2726-49d0-b3de-b795f3e1e33f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:23.238069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8330f-2db2-48b9-ab99-67201217c97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:23.239123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8330f-2db2-48b9-ab99-67201217c97f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:23.239614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8330f-2db2-48b9-ab99-67201217c97f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:23.240116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8330f-2db2-48b9-ab99-67201217c97f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:23.264070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8330f-2db2-48b9-ab99-67201217c97f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:23.265566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac8330f-2db2-48b9-ab99-67201217c97f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3ac8330f-2db2-48b9-ab99-67201217c97f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:23.289471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:23.290577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:23.291487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:23.291487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:23.291487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:23.291487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 354/712] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:23.296306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8330f-2db2-48b9-ab99-67201217c97f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:23.319315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3480908-2726-49d0-b3de-b795f3e1e33f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:23.389888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14188b08-b626-4b5d-9b91-b2fe142bea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:23.390670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14188b08-b626-4b5d-9b91-b2fe142bea1f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:23.390941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14188b08-b626-4b5d-9b91-b2fe142bea1f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:23.391200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14188b08-b626-4b5d-9b91-b2fe142bea1f None admin] PATH_INFO: `/v3/services/network` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:23.391778 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14188b08-b626-4b5d-9b91-b2fe142bea1f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:23.393166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14188b08-b626-4b5d-9b91-b2fe142bea1f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:23.394447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-14188b08-b626-4b5d-9b91-b2fe142bea1f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:23.415313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:23.416450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:23.417313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 359/713] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:23 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:23.427319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14188b08-b626-4b5d-9b91-b2fe142bea1f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:23.443436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:23.469256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8330f-2db2-48b9-ab99-67201217c97f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:23.511446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84193cbb-cd30-428b-a217-cbe7c9f06a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:23.512327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84193cbb-cd30-428b-a217-cbe7c9f06a3c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:23.512599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84193cbb-cd30-428b-a217-cbe7c9f06a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:23.512892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84193cbb-cd30-428b-a217-cbe7c9f06a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:23.513690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84193cbb-cd30-428b-a217-cbe7c9f06a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:23.513988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84193cbb-cd30-428b-a217-cbe7c9f06a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:23.516606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84193cbb-cd30-428b-a217-cbe7c9f06a3c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:23.532627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 355/714] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:23.533995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b782a56-3236-47a5-bd4f-d467c83e3f6a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:23.534277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b782a56-3236-47a5-bd4f-d467c83e3f6a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:23.534544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b782a56-3236-47a5-bd4f-d467c83e3f6a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:23.535998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 356/715] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:23.537436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b782a56-3236-47a5-bd4f-d467c83e3f6a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:23.560814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14188b08-b626-4b5d-9b91-b2fe142bea1f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:23.602965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80c624-5a0f-40d6-86e8-97a273a83922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:23.603887 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80c624-5a0f-40d6-86e8-97a273a83922 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:23.604461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80c624-5a0f-40d6-86e8-97a273a83922 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:23.604737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80c624-5a0f-40d6-86e8-97a273a83922 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:23.605408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80c624-5a0f-40d6-86e8-97a273a83922 None admin] RBAC: Adding query filter params (name=network) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:23.605710 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80c624-5a0f-40d6-86e8-97a273a83922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:23.607081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80c624-5a0f-40d6-86e8-97a273a83922 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:23.613628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 360/716] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:23.614961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851440e2-3190-4c3b-945a-5d92affbce8b None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:23.615564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851440e2-3190-4c3b-945a-5d92affbce8b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:23.615836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851440e2-3190-4c3b-945a-5d92affbce8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:23.638800 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-851440e2-3190-4c3b-945a-5d92affbce8b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:23.640273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-851440e2-3190-4c3b-945a-5d92affbce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:23.670783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b782a56-3236-47a5-bd4f-d467c83e3f6a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:23.689702 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:23.709545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44f3a-14aa-4203-b39f-01e3465869c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:23.710440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44f3a-14aa-4203-b39f-01e3465869c3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:23.710908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44f3a-14aa-4203-b39f-01e3465869c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:23.711322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b44f3a-14aa-4203-b39f-01e3465869c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:23.718952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44f3a-14aa-4203-b39f-01e3465869c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:23.720601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b44f3a-14aa-4203-b39f-01e3465869c3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-91b44f3a-14aa-4203-b39f-01e3465869c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:23.733129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:23.734164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:23.735055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:23.735055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:23.735055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:23.735055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:23.735055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:23.735055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:23.735055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:23.735055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:23.741487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 357/717] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:23.743407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44f3a-14aa-4203-b39f-01e3465869c3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:23.746179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 361/718] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:23.747140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851440e2-3190-4c3b-945a-5d92affbce8b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:23.757933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:23.874337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b44f3a-14aa-4203-b39f-01e3465869c3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:23.912037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d945f58-9900-4659-8706-7ff22155a0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:23.913065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d945f58-9900-4659-8706-7ff22155a0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:23.913552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d945f58-9900-4659-8706-7ff22155a0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:23.914033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d945f58-9900-4659-8706-7ff22155a0d4 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:23.915383 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d945f58-9900-4659-8706-7ff22155a0d4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:23.915901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d945f58-9900-4659-8706-7ff22155a0d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:23.917520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d945f58-9900-4659-8706-7ff22155a0d4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:23.922810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 358/719] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:23.932858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851440e2-3190-4c3b-945a-5d92affbce8b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:23.968575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4200a-5b91-4876-8e88-bc6236a425f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:23.969136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4200a-5b91-4876-8e88-bc6236a425f8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:23.969416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4200a-5b91-4876-8e88-bc6236a425f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:23.969685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4200a-5b91-4876-8e88-bc6236a425f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:23.970396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4200a-5b91-4876-8e88-bc6236a425f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:23.970699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4200a-5b91-4876-8e88-bc6236a425f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:23.972221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4200a-5b91-4876-8e88-bc6236a425f8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:23.987802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 362/720] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:23.994172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d945f58-9900-4659-8706-7ff22155a0d4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:24.063671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4200a-5b91-4876-8e88-bc6236a425f8 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:24.082390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:24.164383 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d945f58-9900-4659-8706-7ff22155a0d4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:24.188299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:24.206577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f12fe-1dc4-48f7-b467-1907d3fb0fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:24.207823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f12fe-1dc4-48f7-b467-1907d3fb0fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:24.208330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f12fe-1dc4-48f7-b467-1907d3fb0fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:24.208977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f12fe-1dc4-48f7-b467-1907d3fb0fe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:24.210489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f12fe-1dc4-48f7-b467-1907d3fb0fe0 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=f58a559951c94d68951da78c82341a00) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:24.211009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f12fe-1dc4-48f7-b467-1907d3fb0fe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:24.213525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f12fe-1dc4-48f7-b467-1907d3fb0fe0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:24.215035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4200a-5b91-4876-8e88-bc6236a425f8 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:24.221053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 359/721] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:23 2022] GET /identity/v3/role_assignments?user.id=f58a559951c94d68951da78c82341a00&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:24.223067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e88bb-e16e-40a3-8509-575492be8b5b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:24.223574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e88bb-e16e-40a3-8509-575492be8b5b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:24.224054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e88bb-e16e-40a3-8509-575492be8b5b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:24.226806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 360/722] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:24.234161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7812c-03ff-45b4-8a8d-4f347dc10776 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:24.234740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7812c-03ff-45b4-8a8d-4f347dc10776 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:24.235192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7812c-03ff-45b4-8a8d-4f347dc10776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:24.256032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb71b87-6938-4875-bfa6-18fff0ff5768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:24.256747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71b87-6938-4875-bfa6-18fff0ff5768 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:24.257148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71b87-6938-4875-bfa6-18fff0ff5768 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:24.257479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb71b87-6938-4875-bfa6-18fff0ff5768 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:24.260714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d6c7812c-03ff-45b4-8a8d-4f347dc10776 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:24.262448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d6c7812c-03ff-45b4-8a8d-4f347dc10776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:24.270938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb71b87-6938-4875-bfa6-18fff0ff5768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-4eb71b87-6938-4875-bfa6-18fff0ff5768 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:24.277156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:24.278342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:24.279403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:24.280373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 363/723] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:24.286570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb71b87-6938-4875-bfa6-18fff0ff5768 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:24.346391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 361/724] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:24.440581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb71b87-6938-4875-bfa6-18fff0ff5768 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:24.466779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:24.480641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1c4cc-2618-41ed-9bf2-5ca2a20f613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:24.481505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1c4cc-2618-41ed-9bf2-5ca2a20f613c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:24.482196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1c4cc-2618-41ed-9bf2-5ca2a20f613c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:24.482900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1c4cc-2618-41ed-9bf2-5ca2a20f613c None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:24.483963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1c4cc-2618-41ed-9bf2-5ca2a20f613c None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:24.484530 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1c4cc-2618-41ed-9bf2-5ca2a20f613c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:24.486176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1c4cc-2618-41ed-9bf2-5ca2a20f613c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:24.496886 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 364/725] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:24.504676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7812c-03ff-45b4-8a8d-4f347dc10776 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:24.557250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:24.614145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7812c-03ff-45b4-8a8d-4f347dc10776 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:24.667143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f66aaf-67d5-46d7-a3c2-05605c209a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:24.667815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f66aaf-67d5-46d7-a3c2-05605c209a82 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:24.668094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f66aaf-67d5-46d7-a3c2-05605c209a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:24.668362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f66aaf-67d5-46d7-a3c2-05605c209a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:24.682754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f66aaf-67d5-46d7-a3c2-05605c209a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:24.684230 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f66aaf-67d5-46d7-a3c2-05605c209a82 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:24.698328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1c4cc-2618-41ed-9bf2-5ca2a20f613c None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-76f66aaf-67d5-46d7-a3c2-05605c209a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:24.705717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:24.706947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:24.708045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:24.708045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:24.708045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:24.711568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 362/726] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:24.714805 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f66aaf-67d5-46d7-a3c2-05605c209a82 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:24.868924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1c4cc-2618-41ed-9bf2-5ca2a20f613c None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:24.873133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:24.899171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f66aaf-67d5-46d7-a3c2-05605c209a82 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:24.929573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c339d-b8fc-4720-afe0-cc8ed643da4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:24.930241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c339d-b8fc-4720-afe0-cc8ed643da4d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:24.930518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c339d-b8fc-4720-afe0-cc8ed643da4d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:24.930784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c339d-b8fc-4720-afe0-cc8ed643da4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:24.950124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c339d-b8fc-4720-afe0-cc8ed643da4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-b88c339d-b8fc-4720-afe0-cc8ed643da4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:24.955280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:24.956215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:24.957057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:24.957980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 365/727] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:24.964666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dace3-041e-420c-b3eb-e71b1e86d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:24.965310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dace3-041e-420c-b3eb-e71b1e86d436 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:24.965586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dace3-041e-420c-b3eb-e71b1e86d436 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:24.965908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732dace3-041e-420c-b3eb-e71b1e86d436 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:24.966558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732dace3-041e-420c-b3eb-e71b1e86d436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:24.966858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732dace3-041e-420c-b3eb-e71b1e86d436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:24.968212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732dace3-041e-420c-b3eb-e71b1e86d436 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:24.970524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c339d-b8fc-4720-afe0-cc8ed643da4d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:24.988824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 363/728] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:25.000831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dace3-041e-420c-b3eb-e71b1e86d436 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:25.057471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:25.119704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732dace3-041e-420c-b3eb-e71b1e86d436 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:25.130339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c339d-b8fc-4720-afe0-cc8ed643da4d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:25.166275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed86a67-7f9d-4ef7-86f1-cf6d4a3c0727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:25.167005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed86a67-7f9d-4ef7-86f1-cf6d4a3c0727 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:25.167224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed86a67-7f9d-4ef7-86f1-cf6d4a3c0727 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:25.167512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed86a67-7f9d-4ef7-86f1-cf6d4a3c0727 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:25.168175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bc491-ce3c-49dd-95c7-b992b6c30970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:25.168790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bc491-ce3c-49dd-95c7-b992b6c30970 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:25.169151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bc491-ce3c-49dd-95c7-b992b6c30970 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:25.169428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bc491-ce3c-49dd-95c7-b992b6c30970 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:25.170193 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bc491-ce3c-49dd-95c7-b992b6c30970 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:25.170604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bc491-ce3c-49dd-95c7-b992b6c30970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:25.171983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bc491-ce3c-49dd-95c7-b992b6c30970 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:25.176026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed86a67-7f9d-4ef7-86f1-cf6d4a3c0727 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:25.178037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed86a67-7f9d-4ef7-86f1-cf6d4a3c0727 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:25.185988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 366/729] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:25.187273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e22ed7-0b29-47f9-b6ba-d9e5c7debe7e None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:25.187562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e22ed7-0b29-47f9-b6ba-d9e5c7debe7e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:25.187838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e22ed7-0b29-47f9-b6ba-d9e5c7debe7e None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:25.189401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 367/730] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:25.190105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e22ed7-0b29-47f9-b6ba-d9e5c7debe7e None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-aed86a67-7f9d-4ef7-86f1-cf6d4a3c0727 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:25.190298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:25.191166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:25.192073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 364/731] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:12:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:25.197038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac972-5f73-457c-b4c8-9cc3bfe5c067 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:25.197314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac972-5f73-457c-b4c8-9cc3bfe5c067 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:25.197586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac972-5f73-457c-b4c8-9cc3bfe5c067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:25.214772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:25.221322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-526ac972-5f73-457c-b4c8-9cc3bfe5c067 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:25.223406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-526ac972-5f73-457c-b4c8-9cc3bfe5c067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:25.320828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e22ed7-0b29-47f9-b6ba-d9e5c7debe7e None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:25.321033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 365/732] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 11:12:25.321895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac972-5f73-457c-b4c8-9cc3bfe5c067 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:25.330294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:25.388704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d314b-5b18-4e67-a412-2f0e315e3a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:25.389280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d314b-5b18-4e67-a412-2f0e315e3a96 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:25.389556 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d314b-5b18-4e67-a412-2f0e315e3a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:25.389822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d314b-5b18-4e67-a412-2f0e315e3a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:25.410681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d314b-5b18-4e67-a412-2f0e315e3a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:25.412131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d314b-5b18-4e67-a412-2f0e315e3a96 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-e37d314b-5b18-4e67-a412-2f0e315e3a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:25.441564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:25.442539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:25.443432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:25.443432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:25.443432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:25.443432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 368/733] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:25.448559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d314b-5b18-4e67-a412-2f0e315e3a96 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:25.520309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac972-5f73-457c-b4c8-9cc3bfe5c067 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:25.571888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d314b-5b18-4e67-a412-2f0e315e3a96 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:25.593496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d252f-6a0c-4e43-8f76-ecfe9325b91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:25.594460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d252f-6a0c-4e43-8f76-ecfe9325b91d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:25.594932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d252f-6a0c-4e43-8f76-ecfe9325b91d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:25.595419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d252f-6a0c-4e43-8f76-ecfe9325b91d None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:25.596343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d252f-6a0c-4e43-8f76-ecfe9325b91d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:25.596869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d252f-6a0c-4e43-8f76-ecfe9325b91d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:25.598499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d252f-6a0c-4e43-8f76-ecfe9325b91d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:25.602613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:25.614182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d740d53-d71d-4ba8-9348-1b7317d048b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:25.618484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d740d53-d71d-4ba8-9348-1b7317d048b8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:25.618484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d740d53-d71d-4ba8-9348-1b7317d048b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:25.618484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d740d53-d71d-4ba8-9348-1b7317d048b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:25.618484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d740d53-d71d-4ba8-9348-1b7317d048b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:25.618484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d740d53-d71d-4ba8-9348-1b7317d048b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:25.622919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d740d53-d71d-4ba8-9348-1b7317d048b8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:25.633539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 366/734] 10.222.0.84 () {56 vars in 1149 bytes} [Wed Aug 24 11:12:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:25.639189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d252f-6a0c-4e43-8f76-ecfe9325b91d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:25.640539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 369/735] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:25.645825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d740d53-d71d-4ba8-9348-1b7317d048b8 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:25.693919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:25.797619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d252f-6a0c-4e43-8f76-ecfe9325b91d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:25.798240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d740d53-d71d-4ba8-9348-1b7317d048b8 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:25.835931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b71395-1c24-43e8-b079-7d178ebd9d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:25.837588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b71395-1c24-43e8-b079-7d178ebd9d3c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:25.838058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b71395-1c24-43e8-b079-7d178ebd9d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:25.838528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b71395-1c24-43e8-b079-7d178ebd9d3c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:25.841913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc90ec-671e-415f-b5a9-d53aa04c5c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:25.842593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc90ec-671e-415f-b5a9-d53aa04c5c98 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:25.842937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc90ec-671e-415f-b5a9-d53aa04c5c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:25.843276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc90ec-671e-415f-b5a9-d53aa04c5c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:25.847103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b71395-1c24-43e8-b079-7d178ebd9d3c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:25.848779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b71395-1c24-43e8-b079-7d178ebd9d3c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-82b71395-1c24-43e8-b079-7d178ebd9d3c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:25.863187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:25.864383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:25.865597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:25.867818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 370/736] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:25.868724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b71395-1c24-43e8-b079-7d178ebd9d3c None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:25.931119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:25.941580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc90ec-671e-415f-b5a9-d53aa04c5c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:25.944468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc90ec-671e-415f-b5a9-d53aa04c5c98 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:25.960200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:25.992002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b71395-1c24-43e8-b079-7d178ebd9d3c None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:26.037279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b13fa-92b1-452d-af69-8931b4d8082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:26.037671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b13fa-92b1-452d-af69-8931b4d8082d None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.037950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b13fa-92b1-452d-af69-8931b4d8082d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.038219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b13fa-92b1-452d-af69-8931b4d8082d None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.038861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b13fa-92b1-452d-af69-8931b4d8082d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:26.040270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b13fa-92b1-452d-af69-8931b4d8082d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:26.071693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 367/737] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 11:12:26.072899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc90ec-671e-415f-b5a9-d53aa04c5c98 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:26.141171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 371/738] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:12:25 2022] POST /identity/v3/services => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:26.142001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b13fa-92b1-452d-af69-8931b4d8082d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:26.194418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:26.206875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc90ec-671e-415f-b5a9-d53aa04c5c98 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:26.283362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b13fa-92b1-452d-af69-8931b4d8082d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:26.284217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a922750-405a-42d0-beec-a8d1061b94c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:26.286035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922750-405a-42d0-beec-a8d1061b94c7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.286509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922750-405a-42d0-beec-a8d1061b94c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.290642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922750-405a-42d0-beec-a8d1061b94c7 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.290642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a922750-405a-42d0-beec-a8d1061b94c7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:26.290642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a922750-405a-42d0-beec-a8d1061b94c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:26.291767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a922750-405a-42d0-beec-a8d1061b94c7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:26.313330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 368/739] 10.222.0.84 () {56 vars in 1135 bytes} [Wed Aug 24 11:12:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:26.314680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21035ea-a309-4cf5-a920-d4bb39ecaf1f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.315074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21035ea-a309-4cf5-a920-d4bb39ecaf1f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.315284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21035ea-a309-4cf5-a920-d4bb39ecaf1f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.316669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 369/740] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:26.321677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21035ea-a309-4cf5-a920-d4bb39ecaf1f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:26.330005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824136a0-79dd-422d-b38e-d5c10972feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:26.330858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824136a0-79dd-422d-b38e-d5c10972feef None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.331332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824136a0-79dd-422d-b38e-d5c10972feef None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.331771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824136a0-79dd-422d-b38e-d5c10972feef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.333900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:26.345478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824136a0-79dd-422d-b38e-d5c10972feef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:26.347209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824136a0-79dd-422d-b38e-d5c10972feef None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-824136a0-79dd-422d-b38e-d5c10972feef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:26.365069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:26.366387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:26.368559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:26.368559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:26.368559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:26.368559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 372/741] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:26.368559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0053c02-fd3e-48b7-8921-963e070828d3 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.368559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0053c02-fd3e-48b7-8921-963e070828d3 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.368559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0053c02-fd3e-48b7-8921-963e070828d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.398138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c0053c02-fd3e-48b7-8921-963e070828d3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:26.399909 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c0053c02-fd3e-48b7-8921-963e070828d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:26.459238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21035ea-a309-4cf5-a920-d4bb39ecaf1f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:26.491686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 373/742] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:26.494571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0053c02-fd3e-48b7-8921-963e070828d3 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:26.520352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02888d95-9859-4a59-a1a0-dd5293069a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:26.521367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02888d95-9859-4a59-a1a0-dd5293069a77 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.523226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02888d95-9859-4a59-a1a0-dd5293069a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.523706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02888d95-9859-4a59-a1a0-dd5293069a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.565179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:26.617115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02888d95-9859-4a59-a1a0-dd5293069a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:26.619246 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02888d95-9859-4a59-a1a0-dd5293069a77 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:26.640181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0053c02-fd3e-48b7-8921-963e070828d3 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:26.666829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:26.692684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bacd3-413a-481b-9469-36549c458105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:26.693806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bacd3-413a-481b-9469-36549c458105 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.694238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bacd3-413a-481b-9469-36549c458105 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.694666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bacd3-413a-481b-9469-36549c458105 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.696475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bacd3-413a-481b-9469-36549c458105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:26.696963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bacd3-413a-481b-9469-36549c458105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:26.698956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bacd3-413a-481b-9469-36549c458105 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:26.716276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 374/743] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:26.718864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f0c9c-f066-406f-b3d1-a056d8f82307 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.719716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f0c9c-f066-406f-b3d1-a056d8f82307 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.720721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f0c9c-f066-406f-b3d1-a056d8f82307 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.723122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 375/744] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:26.724279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f0c9c-f066-406f-b3d1-a056d8f82307 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:26.744653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 370/745] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:26 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 11:12:26.746958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d2b83-d4fd-4ca2-8a54-cd87e4b5f91a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.747427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d2b83-d4fd-4ca2-8a54-cd87e4b5f91a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.747878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d2b83-d4fd-4ca2-8a54-cd87e4b5f91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.773269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-847d2b83-d4fd-4ca2-8a54-cd87e4b5f91a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:26.775196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-847d2b83-d4fd-4ca2-8a54-cd87e4b5f91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:26.837027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:26.880824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 371/746] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 11:12:26.881033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f0c9c-f066-406f-b3d1-a056d8f82307 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:26.881493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d2b83-d4fd-4ca2-8a54-cd87e4b5f91a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:26.918341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a8120-f0f0-4dee-9597-f5bd3d5a74cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:26.918980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a8120-f0f0-4dee-9597-f5bd3d5a74cb None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:26.919253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a8120-f0f0-4dee-9597-f5bd3d5a74cb None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:26.919524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a8120-f0f0-4dee-9597-f5bd3d5a74cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:26.926967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a8120-f0f0-4dee-9597-f5bd3d5a74cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:26.928422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a8120-f0f0-4dee-9597-f5bd3d5a74cb None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-996a8120-f0f0-4dee-9597-f5bd3d5a74cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:26.943496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:26.944465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:26.945378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:26.945378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:26.945378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:26.945378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:26.945378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:26.945378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:26.945378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:26.945378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:26.949160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 376/747] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:26.950022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a8120-f0f0-4dee-9597-f5bd3d5a74cb None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:26.997007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:27.035090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d2b83-d4fd-4ca2-8a54-cd87e4b5f91a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:27.083945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c3e6f-16fd-439f-8b7d-c5131c5118fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:27.084581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c3e6f-16fd-439f-8b7d-c5131c5118fd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:27.084883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c3e6f-16fd-439f-8b7d-c5131c5118fd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:27.085158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c3e6f-16fd-439f-8b7d-c5131c5118fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:27.099885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c3e6f-16fd-439f-8b7d-c5131c5118fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:27.102147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c3e6f-16fd-439f-8b7d-c5131c5118fd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:27.106906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a8120-f0f0-4dee-9597-f5bd3d5a74cb None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-8c3c3e6f-16fd-439f-8b7d-c5131c5118fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:27.125260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:27.126889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:27.127798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:27.127798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:27.127798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:27.133677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 372/748] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:27.134968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c3e6f-16fd-439f-8b7d-c5131c5118fd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:27.173499 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1affaad-9223-4a53-b9ea-855b5cb3d8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:27.175089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1affaad-9223-4a53-b9ea-855b5cb3d8be None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:27.176214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1affaad-9223-4a53-b9ea-855b5cb3d8be None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:27.176943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1affaad-9223-4a53-b9ea-855b5cb3d8be None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:27.178503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1affaad-9223-4a53-b9ea-855b5cb3d8be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:27.179118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:27.181700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1affaad-9223-4a53-b9ea-855b5cb3d8be None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:27.263449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c3e6f-16fd-439f-8b7d-c5131c5118fd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:27.303676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b73f45-daad-4aa6-8491-ac3cbabde15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:27.304284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73f45-daad-4aa6-8491-ac3cbabde15e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:27.304561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73f45-daad-4aa6-8491-ac3cbabde15e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:27.304930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73f45-daad-4aa6-8491-ac3cbabde15e None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:27.305584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b73f45-daad-4aa6-8491-ac3cbabde15e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:27.305886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b73f45-daad-4aa6-8491-ac3cbabde15e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:27.307318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b73f45-daad-4aa6-8491-ac3cbabde15e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:27.326357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 373/749] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:27.326357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b73f45-daad-4aa6-8491-ac3cbabde15e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:27.328111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 377/750] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:12:26 2022] POST /identity/v3/services => generated 240 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:27.329132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1affaad-9223-4a53-b9ea-855b5cb3d8be None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:27.493018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:27.506552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:27.511533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b73f45-daad-4aa6-8491-ac3cbabde15e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:27.525327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1affaad-9223-4a53-b9ea-855b5cb3d8be None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:27.561256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce23731-cd48-47f9-8dfe-7bab4627d22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:27.561864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce23731-cd48-47f9-8dfe-7bab4627d22a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:27.562183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce23731-cd48-47f9-8dfe-7bab4627d22a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:27.562448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce23731-cd48-47f9-8dfe-7bab4627d22a None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:27.563088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce23731-cd48-47f9-8dfe-7bab4627d22a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:27.563386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce23731-cd48-47f9-8dfe-7bab4627d22a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:27.564751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce23731-cd48-47f9-8dfe-7bab4627d22a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:27.577285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed910ce-6348-486f-ac75-f22d62f87489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:27.579389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed910ce-6348-486f-ac75-f22d62f87489 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:27.579389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed910ce-6348-486f-ac75-f22d62f87489 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:27.579942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed910ce-6348-486f-ac75-f22d62f87489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:27.581575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 374/751] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:27.582524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce23731-cd48-47f9-8dfe-7bab4627d22a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:27.603277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed910ce-6348-486f-ac75-f22d62f87489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-3ed910ce-6348-486f-ac75-f22d62f87489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:27.611498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:27.612449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:27.613292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:27.620433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 378/752] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:27.621520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed910ce-6348-486f-ac75-f22d62f87489 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:27.779448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed910ce-6348-486f-ac75-f22d62f87489 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:27.808261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce23731-cd48-47f9-8dfe-7bab4627d22a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:27.837567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511b34f-879f-4a7f-a800-b1383df4fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:27.838221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511b34f-879f-4a7f-a800-b1383df4fd48 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:27.838481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511b34f-879f-4a7f-a800-b1383df4fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:27.838751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511b34f-879f-4a7f-a800-b1383df4fd48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:27.851551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511b34f-879f-4a7f-a800-b1383df4fd48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:27.853081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511b34f-879f-4a7f-a800-b1383df4fd48 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-d511b34f-879f-4a7f-a800-b1383df4fd48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:27.873007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:27.873933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:27.874795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:27.874795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:27.874795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:27.874795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:27.874795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:27.874795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:27.874795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:27.874795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:27.876129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 379/753] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:27.876974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511b34f-879f-4a7f-a800-b1383df4fd48 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:27.888766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cfa42-9186-4f51-9c9a-2a383f061b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:27.889421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cfa42-9186-4f51-9c9a-2a383f061b3b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:27.889701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cfa42-9186-4f51-9c9a-2a383f061b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:27.889970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cfa42-9186-4f51-9c9a-2a383f061b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:27.890707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cfa42-9186-4f51-9c9a-2a383f061b3b None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=164f3bcc6363445da191bd1b78499a53) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:27.891011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cfa42-9186-4f51-9c9a-2a383f061b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:27.892434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cfa42-9186-4f51-9c9a-2a383f061b3b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:27.904111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 375/754] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:27 2022] GET /identity/v3/role_assignments?user.id=164f3bcc6363445da191bd1b78499a53&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:27.905097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cfa42-9186-4f51-9c9a-2a383f061b3b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:27.923868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:27.951175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:28.043746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cfa42-9186-4f51-9c9a-2a383f061b3b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:28.091223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b75b35-acff-45e4-bf73-effe50a5cb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:28.091852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75b35-acff-45e4-bf73-effe50a5cb90 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:28.092136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75b35-acff-45e4-bf73-effe50a5cb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:28.092470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75b35-acff-45e4-bf73-effe50a5cb90 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:28.093206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b75b35-acff-45e4-bf73-effe50a5cb90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:28.093775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b75b35-acff-45e4-bf73-effe50a5cb90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:28.096533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b75b35-acff-45e4-bf73-effe50a5cb90 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:28.115812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 376/755] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:28.121589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b75b35-acff-45e4-bf73-effe50a5cb90 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:28.142471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511b34f-879f-4a7f-a800-b1383df4fd48 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:28.203889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e9e19-e77b-420b-98a5-3d251df59200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:28.204488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e9e19-e77b-420b-98a5-3d251df59200 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:28.204759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e9e19-e77b-420b-98a5-3d251df59200 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:28.205054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e9e19-e77b-420b-98a5-3d251df59200 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:28.205766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e9e19-e77b-420b-98a5-3d251df59200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:28.206061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e9e19-e77b-420b-98a5-3d251df59200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:28.207461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e9e19-e77b-420b-98a5-3d251df59200 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:28.221470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 380/756] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:28.228762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e9e19-e77b-420b-98a5-3d251df59200 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:28.261909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:28.282134 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b75b35-acff-45e4-bf73-effe50a5cb90 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:28.319224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a265de-1dd0-4e21-84a3-e3f9403b3dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:28.319832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a265de-1dd0-4e21-84a3-e3f9403b3dfd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:28.320111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a265de-1dd0-4e21-84a3-e3f9403b3dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:28.320413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a265de-1dd0-4e21-84a3-e3f9403b3dfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:28.321191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a265de-1dd0-4e21-84a3-e3f9403b3dfd None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=1ba54c2312b348b08e0faae745747e7d) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:28.321498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a265de-1dd0-4e21-84a3-e3f9403b3dfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:28.322922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a265de-1dd0-4e21-84a3-e3f9403b3dfd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:28.332928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 377/757] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:28 2022] GET /identity/v3/role_assignments?user.id=1ba54c2312b348b08e0faae745747e7d&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:28.348518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd87b3-dbf4-4b21-8a90-a73c681c1b62 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:28.348885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd87b3-dbf4-4b21-8a90-a73c681c1b62 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:28.349101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd87b3-dbf4-4b21-8a90-a73c681c1b62 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:28.350817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 378/758] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:12:28.357175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c6331-0053-4038-b1bc-20979a145db7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:28.357753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c6331-0053-4038-b1bc-20979a145db7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:28.358014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c6331-0053-4038-b1bc-20979a145db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:28.377776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8a3c6331-0053-4038-b1bc-20979a145db7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:28.379080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8a3c6331-0053-4038-b1bc-20979a145db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:28.445184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:28.478645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e9e19-e77b-420b-98a5-3d251df59200 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:28.506832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 379/759] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:28.552394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e8400-70ee-4197-878d-e0772e5cc57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:28.553073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e8400-70ee-4197-878d-e0772e5cc57e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:28.553348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e8400-70ee-4197-878d-e0772e5cc57e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:28.553618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e8400-70ee-4197-878d-e0772e5cc57e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:28.573304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e8400-70ee-4197-878d-e0772e5cc57e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:28.574689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e8400-70ee-4197-878d-e0772e5cc57e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-7b8e8400-70ee-4197-878d-e0772e5cc57e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:28.593058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:28.594037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:28.595515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:28.595515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:28.595515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:28.595515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 381/760] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:28.599883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c6331-0053-4038-b1bc-20979a145db7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:28.722418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:28.765330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c6331-0053-4038-b1bc-20979a145db7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:28.800920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9341745-3e47-4ab9-ac20-285118caf078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:28.801541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9341745-3e47-4ab9-ac20-285118caf078 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:28.801851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9341745-3e47-4ab9-ac20-285118caf078 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:28.802155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9341745-3e47-4ab9-ac20-285118caf078 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:28.802820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9341745-3e47-4ab9-ac20-285118caf078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:28.803152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9341745-3e47-4ab9-ac20-285118caf078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:28.804540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9341745-3e47-4ab9-ac20-285118caf078 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:28.816652 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 380/761] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:28.821794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e8400-70ee-4197-878d-e0772e5cc57e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:28.947279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9341745-3e47-4ab9-ac20-285118caf078 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:28.977224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e8400-70ee-4197-878d-e0772e5cc57e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:29.006749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:29.020266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3007b-4347-4986-a702-d31f4bc89b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:29.020914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3007b-4347-4986-a702-d31f4bc89b17 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:29.021186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3007b-4347-4986-a702-d31f4bc89b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:29.021447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3007b-4347-4986-a702-d31f4bc89b17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:29.039979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3007b-4347-4986-a702-d31f4bc89b17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:29.041470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3007b-4347-4986-a702-d31f4bc89b17 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-9cb3007b-4347-4986-a702-d31f4bc89b17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:29.063157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:29.064117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:29.065100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 382/762] 10.222.0.84 () {56 vars in 1093 bytes} [Wed Aug 24 11:12:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:29.073289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3007b-4347-4986-a702-d31f4bc89b17 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:29.172506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9341745-3e47-4ab9-ac20-285118caf078 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:29.221067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3007b-4347-4986-a702-d31f4bc89b17 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:29.241135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:29.254650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab5a5c-bfe4-4dec-ac78-63434c79ee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:29.255385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab5a5c-bfe4-4dec-ac78-63434c79ee17 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:29.255696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab5a5c-bfe4-4dec-ac78-63434c79ee17 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:29.256034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab5a5c-bfe4-4dec-ac78-63434c79ee17 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:29.256619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab5a5c-bfe4-4dec-ac78-63434c79ee17 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:29.258634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab5a5c-bfe4-4dec-ac78-63434c79ee17 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:29.268685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddf7fc-38c6-41b8-bcf9-b00e4dd36f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:29.269318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddf7fc-38c6-41b8-bcf9-b00e4dd36f2b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:29.269591 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddf7fc-38c6-41b8-bcf9-b00e4dd36f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:29.269915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddf7fc-38c6-41b8-bcf9-b00e4dd36f2b None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:29.270638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddf7fc-38c6-41b8-bcf9-b00e4dd36f2b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:29.270933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddf7fc-38c6-41b8-bcf9-b00e4dd36f2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a6ab5a5c-bfe4-4dec-ac78-63434c79ee17 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:29.271874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:29.272782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:12:29.274193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:29.274907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ddf7fc-38c6-41b8-bcf9-b00e4dd36f2b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:29.281137 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 381/763] 10.222.0.84 () {56 vars in 1129 bytes} [Wed Aug 24 11:12:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:29.282040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab5a5c-bfe4-4dec-ac78-63434c79ee17 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:29.292382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 383/764] 10.222.0.84 () {56 vars in 1133 bytes} [Wed Aug 24 11:12:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:29.302397 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddf7fc-38c6-41b8-bcf9-b00e4dd36f2b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:29.393161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:29.470904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab5a5c-bfe4-4dec-ac78-63434c79ee17 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:29.471176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddf7fc-38c6-41b8-bcf9-b00e4dd36f2b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:29.510229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70f7f-936d-4568-be2b-f94a3deed23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:29.511084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70f7f-936d-4568-be2b-f94a3deed23d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:29.511361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70f7f-936d-4568-be2b-f94a3deed23d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:29.511628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed70f7f-936d-4568-be2b-f94a3deed23d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:29.524484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed70f7f-936d-4568-be2b-f94a3deed23d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:29.526574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed70f7f-936d-4568-be2b-f94a3deed23d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-4ed70f7f-936d-4568-be2b-f94a3deed23d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:29.544641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:29.545633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:29.546486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:29.546486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:29.546486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:29.546486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 384/765] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:29.551414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70f7f-936d-4568-be2b-f94a3deed23d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:29.574844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6a54a-9b08-4807-af39-1ad415d36d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:29.575457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6a54a-9b08-4807-af39-1ad415d36d95 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:29.575731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6a54a-9b08-4807-af39-1ad415d36d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:29.575998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6a54a-9b08-4807-af39-1ad415d36d95 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:29.576635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6a54a-9b08-4807-af39-1ad415d36d95 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:29.576974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6a54a-9b08-4807-af39-1ad415d36d95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:29.579927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6a54a-9b08-4807-af39-1ad415d36d95 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:29.592012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 382/766] 10.222.0.84 () {56 vars in 1124 bytes} [Wed Aug 24 11:12:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:29.598783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6a54a-9b08-4807-af39-1ad415d36d95 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:29.694437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:29.747840 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6a54a-9b08-4807-af39-1ad415d36d95 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:29.757505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:29.763118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed70f7f-936d-4568-be2b-f94a3deed23d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:29.794288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc3fe8-706a-4d64-922c-cfac312a44a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:29.794892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc3fe8-706a-4d64-922c-cfac312a44a3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:29.795164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc3fe8-706a-4d64-922c-cfac312a44a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:29.795428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc3fe8-706a-4d64-922c-cfac312a44a3 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:29.796047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc3fe8-706a-4d64-922c-cfac312a44a3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:29.796341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc3fe8-706a-4d64-922c-cfac312a44a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:29.797783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc3fe8-706a-4d64-922c-cfac312a44a3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:29.801685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0490691-520b-414c-b9ba-a4cab9b2ada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:29.802574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0490691-520b-414c-b9ba-a4cab9b2ada8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:29.803103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 383/767] 10.222.0.84 () {56 vars in 1124 bytes} [Wed Aug 24 11:12:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:29.803326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0490691-520b-414c-b9ba-a4cab9b2ada8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:29.803838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0490691-520b-414c-b9ba-a4cab9b2ada8 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:29.804250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32356366-9e98-4271-8d2f-883021d730d5 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:29.804529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32356366-9e98-4271-8d2f-883021d730d5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:29.805424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32356366-9e98-4271-8d2f-883021d730d5 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:29.806024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0490691-520b-414c-b9ba-a4cab9b2ada8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:29.806518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0490691-520b-414c-b9ba-a4cab9b2ada8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:29.808447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0490691-520b-414c-b9ba-a4cab9b2ada8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:29.810874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 384/768] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:29.811949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32356366-9e98-4271-8d2f-883021d730d5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:29.823559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 385/769] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:29.825186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5370e-b92e-4743-925c-fe44d8ae06eb None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:29.825668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5370e-b92e-4743-925c-fe44d8ae06eb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:29.826142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5370e-b92e-4743-925c-fe44d8ae06eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:29.847077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-06a5370e-b92e-4743-925c-fe44d8ae06eb None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:29.849170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-06a5370e-b92e-4743-925c-fe44d8ae06eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:29.936519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 386/770] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:29.938167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5370e-b92e-4743-925c-fe44d8ae06eb None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:29.957594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32356366-9e98-4271-8d2f-883021d730d5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:30.001815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:30.033284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bece8a-a215-4a6a-8103-a66d5ba13367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:30.033940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bece8a-a215-4a6a-8103-a66d5ba13367 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.034219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bece8a-a215-4a6a-8103-a66d5ba13367 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.034491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bece8a-a215-4a6a-8103-a66d5ba13367 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.035193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bece8a-a215-4a6a-8103-a66d5ba13367 None admin] RBAC: Adding query filter params (interface=public, service_id=658db7908a564b5796c75966eb595448, region_id=RegionOne) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:30.035499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bece8a-a215-4a6a-8103-a66d5ba13367 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:30.036894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bece8a-a215-4a6a-8103-a66d5ba13367 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:30.059713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 385/771] 10.222.0.84 () {56 vars in 1251 bytes} [Wed Aug 24 11:12:29 2022] GET /identity/v3/endpoints?service_id=658db7908a564b5796c75966eb595448&interface=public®ion_id=RegionOne => generated 194 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:30.062462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bece8a-a215-4a6a-8103-a66d5ba13367 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:30.105395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:30.133446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5370e-b92e-4743-925c-fe44d8ae06eb None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:30.181149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bece8a-a215-4a6a-8103-a66d5ba13367 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:30.202085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7317b2-e611-4400-aca7-4a64ea63f199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:30.202772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7317b2-e611-4400-aca7-4a64ea63f199 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.203096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7317b2-e611-4400-aca7-4a64ea63f199 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.203369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7317b2-e611-4400-aca7-4a64ea63f199 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.213580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7317b2-e611-4400-aca7-4a64ea63f199 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:30.215025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7317b2-e611-4400-aca7-4a64ea63f199 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:30.220067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db050f-59ea-4e7e-b1b4-746ffff22120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:30.220661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db050f-59ea-4e7e-b1b4-746ffff22120 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.221036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db050f-59ea-4e7e-b1b4-746ffff22120 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.221329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db050f-59ea-4e7e-b1b4-746ffff22120 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.221823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db050f-59ea-4e7e-b1b4-746ffff22120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:30.223250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db050f-59ea-4e7e-b1b4-746ffff22120 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-6e7317b2-e611-4400-aca7-4a64ea63f199 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:30.227190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:30.228093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:30.234459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:30.234459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:30.234459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:30.234459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:30.234459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:30.234459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:30.234459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:30.234459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:30.234459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 387/772] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:30.234958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 386/773] 10.222.0.84 () {56 vars in 1087 bytes} [Wed Aug 24 11:12:30 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:30.235045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5504b8-6b86-484a-9835-96a475b23fbe None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.235045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5504b8-6b86-484a-9835-96a475b23fbe None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.235045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5504b8-6b86-484a-9835-96a475b23fbe None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.235247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db050f-59ea-4e7e-b1b4-746ffff22120 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:30.235316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 388/774] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:30.237126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5504b8-6b86-484a-9835-96a475b23fbe None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:30.328395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:30.381937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5504b8-6b86-484a-9835-96a475b23fbe None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:30.385756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db050f-59ea-4e7e-b1b4-746ffff22120 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:30.423112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b492e9c-834d-427b-976a-0320b45481e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:30.423982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b492e9c-834d-427b-976a-0320b45481e8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.424510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b492e9c-834d-427b-976a-0320b45481e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.425090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b492e9c-834d-427b-976a-0320b45481e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.425983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b492e9c-834d-427b-976a-0320b45481e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:30.426431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b492e9c-834d-427b-976a-0320b45481e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:30.428201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b492e9c-834d-427b-976a-0320b45481e8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:30.428405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dac1b2-0884-49d1-b6cf-5ca538be61f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:30.429320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dac1b2-0884-49d1-b6cf-5ca538be61f5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.429806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dac1b2-0884-49d1-b6cf-5ca538be61f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.430492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dac1b2-0884-49d1-b6cf-5ca538be61f5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.431456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dac1b2-0884-49d1-b6cf-5ca538be61f5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:30.433046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dac1b2-0884-49d1-b6cf-5ca538be61f5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:30.441231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 389/775] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:30.442450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c2dec-9823-4175-bb17-fadb23e5f57b None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.442732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c2dec-9823-4175-bb17-fadb23e5f57b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a7dac1b2-0884-49d1-b6cf-5ca538be61f5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:30.442972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:30.443902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:12:30.444718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:30.445572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c2dec-9823-4175-bb17-fadb23e5f57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.447845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 387/776] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:30 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:30.450863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dac1b2-0884-49d1-b6cf-5ca538be61f5 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:30.461978 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:30.473339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-2f5c2dec-9823-4175-bb17-fadb23e5f57b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:30.474840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2f5c2dec-9823-4175-bb17-fadb23e5f57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:30.554292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 390/777] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:30.556225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c2dec-9823-4175-bb17-fadb23e5f57b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:30.600791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dac1b2-0884-49d1-b6cf-5ca538be61f5 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:30.619751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:30.656474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db807c4-9d4e-4675-9220-4cd94ec16de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:30.656474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db807c4-9d4e-4675-9220-4cd94ec16de4 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.656474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db807c4-9d4e-4675-9220-4cd94ec16de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.656474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db807c4-9d4e-4675-9220-4cd94ec16de4 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/f58a559951c94d68951da78c82341a00/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.658216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db807c4-9d4e-4675-9220-4cd94ec16de4 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=f58a559951c94d68951da78c82341a00, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:30.659665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db807c4-9d4e-4675-9220-4cd94ec16de4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:30.682319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c2dec-9823-4175-bb17-fadb23e5f57b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:30.717407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddeeaf-1609-4ae1-971d-2cfef49b1568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:30.717998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddeeaf-1609-4ae1-971d-2cfef49b1568 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.718272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddeeaf-1609-4ae1-971d-2cfef49b1568 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.718535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddeeaf-1609-4ae1-971d-2cfef49b1568 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.719175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddeeaf-1609-4ae1-971d-2cfef49b1568 None admin] RBAC: Adding query filter params (name=network) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:30.719469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddeeaf-1609-4ae1-971d-2cfef49b1568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:30.720850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddeeaf-1609-4ae1-971d-2cfef49b1568 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:30.723291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:30.727705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 391/778] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:30.732866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8518-c269-423b-a839-47d2a6b360c2 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.732866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8518-c269-423b-a839-47d2a6b360c2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.732866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8518-c269-423b-a839-47d2a6b360c2 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.735118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 392/779] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:30.735894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8518-c269-423b-a839-47d2a6b360c2 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:30.752019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 388/780] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:12:30 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/f58a559951c94d68951da78c82341a00/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:30.753363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81ef5a-cd96-4a1b-9abb-05cee79749cd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.753649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81ef5a-cd96-4a1b-9abb-05cee79749cd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.753920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81ef5a-cd96-4a1b-9abb-05cee79749cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.775407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5f81ef5a-cd96-4a1b-9abb-05cee79749cd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:30.776618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5f81ef5a-cd96-4a1b-9abb-05cee79749cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:30.860948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 389/781] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:30.888347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8518-c269-423b-a839-47d2a6b360c2 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:30.909711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81ef5a-cd96-4a1b-9abb-05cee79749cd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:30.936232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659fc2c-8db7-4d9e-95ed-62966847fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:30.936865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659fc2c-8db7-4d9e-95ed-62966847fd63 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:30.937203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659fc2c-8db7-4d9e-95ed-62966847fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:30.937477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659fc2c-8db7-4d9e-95ed-62966847fd63 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:30.938111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659fc2c-8db7-4d9e-95ed-62966847fd63 None admin] RBAC: Adding query filter params (type=network) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:30.938402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659fc2c-8db7-4d9e-95ed-62966847fd63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:30.939768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659fc2c-8db7-4d9e-95ed-62966847fd63 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:30.949100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 393/782] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:30.957973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659fc2c-8db7-4d9e-95ed-62966847fd63 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.041238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:31.062047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81ef5a-cd96-4a1b-9abb-05cee79749cd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.063554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:31.093220 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659fc2c-8db7-4d9e-95ed-62966847fd63 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.103274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad03c6-57a9-4b9f-bce0-d43735acb9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.103830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad03c6-57a9-4b9f-bce0-d43735acb9bf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.104104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad03c6-57a9-4b9f-bce0-d43735acb9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.104370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad03c6-57a9-4b9f-bce0-d43735acb9bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.115164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad03c6-57a9-4b9f-bce0-d43735acb9bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-54ad03c6-57a9-4b9f-bce0-d43735acb9bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:31.120108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:31.121389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:31.122594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 390/783] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:31.126865 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad03c6-57a9-4b9f-bce0-d43735acb9bf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.137004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37f07-125c-41c8-bc3a-ecb672229edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.138162 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37f07-125c-41c8-bc3a-ecb672229edc None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.138657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37f07-125c-41c8-bc3a-ecb672229edc None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.139105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37f07-125c-41c8-bc3a-ecb672229edc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.139968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d37f07-125c-41c8-bc3a-ecb672229edc None admin] RBAC: Adding query filter params (interface=public, service_id=89071b8062754771a3315b4816bf0cdf, region_id=RegionOne) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:31.140442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d37f07-125c-41c8-bc3a-ecb672229edc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:31.142056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d37f07-125c-41c8-bc3a-ecb672229edc None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:31.150829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 394/784] 10.222.0.84 () {56 vars in 1251 bytes} [Wed Aug 24 11:12:30 2022] GET /identity/v3/endpoints?service_id=89071b8062754771a3315b4816bf0cdf&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:31.157724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37f07-125c-41c8-bc3a-ecb672229edc None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.242022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad03c6-57a9-4b9f-bce0-d43735acb9bf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.266046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37f07-125c-41c8-bc3a-ecb672229edc None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.280539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929f7d2-7488-41e5-8a70-7e5c2e6bca9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.281456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929f7d2-7488-41e5-8a70-7e5c2e6bca9f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.281743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929f7d2-7488-41e5-8a70-7e5c2e6bca9f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.282018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929f7d2-7488-41e5-8a70-7e5c2e6bca9f None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.282759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929f7d2-7488-41e5-8a70-7e5c2e6bca9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:31.282976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929f7d2-7488-41e5-8a70-7e5c2e6bca9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:31.284346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929f7d2-7488-41e5-8a70-7e5c2e6bca9f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:31.293847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 391/785] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:31.294836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929f7d2-7488-41e5-8a70-7e5c2e6bca9f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.296113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:31.305682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec7d24-e20f-4b06-891d-e51733951658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.306512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec7d24-e20f-4b06-891d-e51733951658 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.306885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec7d24-e20f-4b06-891d-e51733951658 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.307270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec7d24-e20f-4b06-891d-e51733951658 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.307974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec7d24-e20f-4b06-891d-e51733951658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:31.309474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec7d24-e20f-4b06-891d-e51733951658 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:31.315366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 395/786] 10.222.0.84 () {56 vars in 1087 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/services => generated 772 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:31.316519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec7d24-e20f-4b06-891d-e51733951658 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.326699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:31.417835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929f7d2-7488-41e5-8a70-7e5c2e6bca9f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.432611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec7d24-e20f-4b06-891d-e51733951658 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.460365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a977785-8b75-4dd5-a0da-2fca6c544300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.461397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a977785-8b75-4dd5-a0da-2fca6c544300 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.462033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a977785-8b75-4dd5-a0da-2fca6c544300 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.462521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a977785-8b75-4dd5-a0da-2fca6c544300 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.472846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bc551-c4fb-4f2d-bb9e-ce33d3aa0726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.475649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bc551-c4fb-4f2d-bb9e-ce33d3aa0726 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.475649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bc551-c4fb-4f2d-bb9e-ce33d3aa0726 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.475649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bc551-c4fb-4f2d-bb9e-ce33d3aa0726 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.476279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a977785-8b75-4dd5-a0da-2fca6c544300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-8a977785-8b75-4dd5-a0da-2fca6c544300 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:31.483023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:31.483967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:31.485271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 392/787] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:31.492961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a977785-8b75-4dd5-a0da-2fca6c544300 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.493297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1bc551-c4fb-4f2d-bb9e-ce33d3aa0726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:31.494843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1bc551-c4fb-4f2d-bb9e-ce33d3aa0726 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-bd1bc551-c4fb-4f2d-bb9e-ce33d3aa0726 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:31.510488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:31.511482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:31.513581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:31.513581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:31.513581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:31.513581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 396/788] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:31.517508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bc551-c4fb-4f2d-bb9e-ce33d3aa0726 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.572614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:31.627258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bc551-c4fb-4f2d-bb9e-ce33d3aa0726 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.647830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:31.668582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a977785-8b75-4dd5-a0da-2fca6c544300 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.668915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046e0c7-8733-4480-b7c2-2ebb7c4a864d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.669528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046e0c7-8733-4480-b7c2-2ebb7c4a864d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.669813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046e0c7-8733-4480-b7c2-2ebb7c4a864d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.670083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046e0c7-8733-4480-b7c2-2ebb7c4a864d None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.670726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046e0c7-8733-4480-b7c2-2ebb7c4a864d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:31.671027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046e0c7-8733-4480-b7c2-2ebb7c4a864d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:31.672401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046e0c7-8733-4480-b7c2-2ebb7c4a864d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:31.686672 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 397/789] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:31.692060 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046e0c7-8733-4480-b7c2-2ebb7c4a864d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.706052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e0bd2-08fd-4db7-86fd-6841925b00f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.706709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0bd2-08fd-4db7-86fd-6841925b00f1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.706985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0bd2-08fd-4db7-86fd-6841925b00f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.707308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e0bd2-08fd-4db7-86fd-6841925b00f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.707971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e0bd2-08fd-4db7-86fd-6841925b00f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:31.708316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e0bd2-08fd-4db7-86fd-6841925b00f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:31.709766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e0bd2-08fd-4db7-86fd-6841925b00f1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:31.720361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 393/790] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:31.727123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e0bd2-08fd-4db7-86fd-6841925b00f1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.822160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046e0c7-8733-4480-b7c2-2ebb7c4a864d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.825351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:31.862548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f14a9-24a5-4f72-a758-c34dbb486154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.863563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f14a9-24a5-4f72-a758-c34dbb486154 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.870139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f14a9-24a5-4f72-a758-c34dbb486154 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.870139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f14a9-24a5-4f72-a758-c34dbb486154 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:31.873152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f14a9-24a5-4f72-a758-c34dbb486154 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:31.874935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f14a9-24a5-4f72-a758-c34dbb486154 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-0c8f14a9-24a5-4f72-a758-c34dbb486154 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:31.887760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:31.888828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:31.889747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:31.892293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 398/791] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:31.899294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f14a9-24a5-4f72-a758-c34dbb486154 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:31.954196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e0bd2-08fd-4db7-86fd-6841925b00f1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:31.991170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d8c5-c0e4-425e-be53-9b99be71fa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:31.991773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d8c5-c0e4-425e-be53-9b99be71fa74 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:31.992050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d8c5-c0e4-425e-be53-9b99be71fa74 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:31.992420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d8c5-c0e4-425e-be53-9b99be71fa74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.003442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997d8c5-c0e4-425e-be53-9b99be71fa74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:32.004919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997d8c5-c0e4-425e-be53-9b99be71fa74 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:32.012042 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:32.016123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f14a9-24a5-4f72-a758-c34dbb486154 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d997d8c5-c0e4-425e-be53-9b99be71fa74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:32.019910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:32.030721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:32.031596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:32.031596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:32.031596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:32.031596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 394/792] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:32.034531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d8c5-c0e4-425e-be53-9b99be71fa74 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:32.078121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce8971-ab28-4e55-b9a8-292ae6574497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:32.079263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce8971-ab28-4e55-b9a8-292ae6574497 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.080911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce8971-ab28-4e55-b9a8-292ae6574497 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.081364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce8971-ab28-4e55-b9a8-292ae6574497 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.082263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce8971-ab28-4e55-b9a8-292ae6574497 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:32.083499 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce8971-ab28-4e55-b9a8-292ae6574497 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:32.087784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce8971-ab28-4e55-b9a8-292ae6574497 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:32.121696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 399/793] 10.222.0.84 () {56 vars in 1149 bytes} [Wed Aug 24 11:12:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:32.128390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce8971-ab28-4e55-b9a8-292ae6574497 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:32.165264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:32.248124 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d8c5-c0e4-425e-be53-9b99be71fa74 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:32.253803 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce8971-ab28-4e55-b9a8-292ae6574497 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:32.296410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce58a3-0c23-4e89-82fa-987760751cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:32.297085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce58a3-0c23-4e89-82fa-987760751cfa None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.297363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce58a3-0c23-4e89-82fa-987760751cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.297631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce58a3-0c23-4e89-82fa-987760751cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.311372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce58a3-0c23-4e89-82fa-987760751cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:32.312865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce58a3-0c23-4e89-82fa-987760751cfa None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:32.332531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc291098-bf3d-40b2-abdc-1744c50a36b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:32.333164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc291098-bf3d-40b2-abdc-1744c50a36b1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.333442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc291098-bf3d-40b2-abdc-1744c50a36b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.333710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc291098-bf3d-40b2-abdc-1744c50a36b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.334518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc291098-bf3d-40b2-abdc-1744c50a36b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:32.334826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc291098-bf3d-40b2-abdc-1744c50a36b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:32.337319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc291098-bf3d-40b2-abdc-1744c50a36b1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-a4ce58a3-0c23-4e89-82fa-987760751cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:32.343724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:32.344676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:32.345598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:32.345598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:32.345598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:32.345598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 400/794] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:32.353335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce58a3-0c23-4e89-82fa-987760751cfa None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:32.374985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 395/795] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:32.382097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc291098-bf3d-40b2-abdc-1744c50a36b1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:32.452876 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:32.497964 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc291098-bf3d-40b2-abdc-1744c50a36b1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:32.522509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:32.536291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ef7c7-b4ef-4d10-902d-787bfdac2fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:32.537292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ef7c7-b4ef-4d10-902d-787bfdac2fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.537777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ef7c7-b4ef-4d10-902d-787bfdac2fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.538298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ef7c7-b4ef-4d10-902d-787bfdac2fa4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.546622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ef7c7-b4ef-4d10-902d-787bfdac2fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:32.548393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ef7c7-b4ef-4d10-902d-787bfdac2fa4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-084ef7c7-b4ef-4d10-902d-787bfdac2fa4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:32.563769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:32.564754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:32.566111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:32.567362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce58a3-0c23-4e89-82fa-987760751cfa None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:32.568759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 396/796] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:32.576244 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ef7c7-b4ef-4d10-902d-787bfdac2fa4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:32.656998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcfb5eb-326d-4acf-aea7-79c286e666d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:32.657646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfb5eb-326d-4acf-aea7-79c286e666d5 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.657913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfb5eb-326d-4acf-aea7-79c286e666d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.658182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcfb5eb-326d-4acf-aea7-79c286e666d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.658831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfb5eb-326d-4acf-aea7-79c286e666d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:32.659132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfb5eb-326d-4acf-aea7-79c286e666d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:32.660597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcfb5eb-326d-4acf-aea7-79c286e666d5 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:32.690614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ef7c7-b4ef-4d10-902d-787bfdac2fa4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:32.691626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 401/797] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:32.692941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda08d02-109e-45bc-ba16-c8acf79a2d31 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.693225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda08d02-109e-45bc-ba16-c8acf79a2d31 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.693493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda08d02-109e-45bc-ba16-c8acf79a2d31 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.694984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 402/798] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:32.700847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda08d02-109e-45bc-ba16-c8acf79a2d31 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:32.713175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:32.731550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2f67f-ec4d-4a14-8877-da08b7aed7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:32.732156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2f67f-ec4d-4a14-8877-da08b7aed7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.732845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2f67f-ec4d-4a14-8877-da08b7aed7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.733054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2f67f-ec4d-4a14-8877-da08b7aed7b5 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.733994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2f67f-ec4d-4a14-8877-da08b7aed7b5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:32.734358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2f67f-ec4d-4a14-8877-da08b7aed7b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:32.735849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2f67f-ec4d-4a14-8877-da08b7aed7b5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:32.756498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 397/799] 10.222.0.84 () {56 vars in 1135 bytes} [Wed Aug 24 11:12:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:32.762010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864a6f4-70a7-4d56-9677-9b8d04f7c49c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.762295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864a6f4-70a7-4d56-9677-9b8d04f7c49c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.762570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864a6f4-70a7-4d56-9677-9b8d04f7c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.783453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-3864a6f4-70a7-4d56-9677-9b8d04f7c49c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:32.784598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3864a6f4-70a7-4d56-9677-9b8d04f7c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:32.814384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda08d02-109e-45bc-ba16-c8acf79a2d31 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:32.852078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f5d93-477e-42eb-96de-2c2ae4411af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:32.853033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f5d93-477e-42eb-96de-2c2ae4411af0 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:32.853404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f5d93-477e-42eb-96de-2c2ae4411af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:32.853861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f5d93-477e-42eb-96de-2c2ae4411af0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:32.856292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:32.862337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f5d93-477e-42eb-96de-2c2ae4411af0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:32.862867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 398/800] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:32.863786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864a6f4-70a7-4d56-9677-9b8d04f7c49c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:32.870280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f5d93-477e-42eb-96de-2c2ae4411af0 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-c47f5d93-477e-42eb-96de-2c2ae4411af0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:32.886847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:32.887726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:32.888596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:32.888596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:32.888596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:32.888596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:32.888596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:32.888596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:32.888596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:32.888596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:32.892149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 403/801] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:32.897721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f5d93-477e-42eb-96de-2c2ae4411af0 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:33.052754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864a6f4-70a7-4d56-9677-9b8d04f7c49c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:33.065130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:33.099856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f5d93-477e-42eb-96de-2c2ae4411af0 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:33.121625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59459dc6-8085-43e5-84eb-4d77d756cdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:33.122265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59459dc6-8085-43e5-84eb-4d77d756cdfa None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.122542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59459dc6-8085-43e5-84eb-4d77d756cdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.122814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59459dc6-8085-43e5-84eb-4d77d756cdfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.134583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59459dc6-8085-43e5-84eb-4d77d756cdfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:33.136054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59459dc6-8085-43e5-84eb-4d77d756cdfa None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-59459dc6-8085-43e5-84eb-4d77d756cdfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:33.151608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:33.152563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:33.153505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:33.153505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:33.153505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:33.157139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 399/802] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:33.158948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59459dc6-8085-43e5-84eb-4d77d756cdfa None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:33.189522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53323942-f2c7-414c-acb5-6b8602114f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:33.190850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53323942-f2c7-414c-acb5-6b8602114f8a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.191332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53323942-f2c7-414c-acb5-6b8602114f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.191809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53323942-f2c7-414c-acb5-6b8602114f8a None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.192722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53323942-f2c7-414c-acb5-6b8602114f8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:33.193251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53323942-f2c7-414c-acb5-6b8602114f8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:33.194930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53323942-f2c7-414c-acb5-6b8602114f8a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:33.208504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 404/803] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:33.209421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53323942-f2c7-414c-acb5-6b8602114f8a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:33.269772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59459dc6-8085-43e5-84eb-4d77d756cdfa None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:33.276791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:33.310256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fb9f0-c13c-4b49-bf21-4087252dd578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:33.310873 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fb9f0-c13c-4b49-bf21-4087252dd578 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.311162 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fb9f0-c13c-4b49-bf21-4087252dd578 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.311432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fb9f0-c13c-4b49-bf21-4087252dd578 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.312079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fb9f0-c13c-4b49-bf21-4087252dd578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:33.312381 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fb9f0-c13c-4b49-bf21-4087252dd578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:33.313790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81fb9f0-c13c-4b49-bf21-4087252dd578 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:33.326993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 400/804] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:33.327780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fb9f0-c13c-4b49-bf21-4087252dd578 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:33.331181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53323942-f2c7-414c-acb5-6b8602114f8a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:33.349701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:33.370611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42eb60-aac8-42c2-b697-38cf74fb955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:33.371244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42eb60-aac8-42c2-b697-38cf74fb955a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.371524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42eb60-aac8-42c2-b697-38cf74fb955a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.371796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42eb60-aac8-42c2-b697-38cf74fb955a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.372376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42eb60-aac8-42c2-b697-38cf74fb955a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:33.373771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42eb60-aac8-42c2-b697-38cf74fb955a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-5f42eb60-aac8-42c2-b697-38cf74fb955a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:33.382619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:33.384170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 11:12:33.385149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:33.385883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 405/805] 10.222.0.84 () {56 vars in 1129 bytes} [Wed Aug 24 11:12:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:33.386455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3903f-041b-407b-bfc1-cfba181621e7 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.386781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3903f-041b-407b-bfc1-cfba181621e7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.387055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3903f-041b-407b-bfc1-cfba181621e7 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.392391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 406/806] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:33.392391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3903f-041b-407b-bfc1-cfba181621e7 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:33.444610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fb9f0-c13c-4b49-bf21-4087252dd578 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:33.483391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f471c-a927-404e-8844-6c6f0aa526e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:33.484051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f471c-a927-404e-8844-6c6f0aa526e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.484325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f471c-a927-404e-8844-6c6f0aa526e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.484594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f471c-a927-404e-8844-6c6f0aa526e8 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/164f3bcc6363445da191bd1b78499a53/roles/69cd1c433d70467a91f4a9f47382a412` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.501783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f471c-a927-404e-8844-6c6f0aa526e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=164f3bcc6363445da191bd1b78499a53, role_id=69cd1c433d70467a91f4a9f47382a412)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:33.503220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f471c-a927-404e-8844-6c6f0aa526e8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:33.509014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3903f-041b-407b-bfc1-cfba181621e7 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:33.526550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:33.543219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fd5da-19ed-44af-ad78-c6c21a06a34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:33.543931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fd5da-19ed-44af-ad78-c6c21a06a34c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.544202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fd5da-19ed-44af-ad78-c6c21a06a34c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.544463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fd5da-19ed-44af-ad78-c6c21a06a34c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.550895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fd5da-19ed-44af-ad78-c6c21a06a34c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:33.552347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fd5da-19ed-44af-ad78-c6c21a06a34c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-0f2fd5da-19ed-44af-ad78-c6c21a06a34c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:33.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:33.565161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 407/807] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:33.566022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fd5da-19ed-44af-ad78-c6c21a06a34c None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:33.581315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 401/808] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:12:33 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/164f3bcc6363445da191bd1b78499a53/roles/69cd1c433d70467a91f4a9f47382a412 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:33.583673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4c11c-23e1-4333-a632-121955c9c98d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.583673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4c11c-23e1-4333-a632-121955c9c98d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.583673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4c11c-23e1-4333-a632-121955c9c98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.604054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-bca4c11c-23e1-4333-a632-121955c9c98d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:33.605423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bca4c11c-23e1-4333-a632-121955c9c98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:33.682583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:33.692963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 402/809] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:33.694379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4c11c-23e1-4333-a632-121955c9c98d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:33.725221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fd5da-19ed-44af-ad78-c6c21a06a34c None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:33.777285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c2655-22e9-4dda-b2fc-e95250c8453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:33.778378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c2655-22e9-4dda-b2fc-e95250c8453a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.778835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c2655-22e9-4dda-b2fc-e95250c8453a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.779612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c2655-22e9-4dda-b2fc-e95250c8453a None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.780694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c2655-22e9-4dda-b2fc-e95250c8453a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:33.781235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c2655-22e9-4dda-b2fc-e95250c8453a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:33.783515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c2655-22e9-4dda-b2fc-e95250c8453a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:33.790402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 408/810] 10.222.0.84 () {56 vars in 1124 bytes} [Wed Aug 24 11:12:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:33.793115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057fce5-d09e-4bbb-9261-481ae17597f6 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.793398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057fce5-d09e-4bbb-9261-481ae17597f6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.793668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057fce5-d09e-4bbb-9261-481ae17597f6 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.795716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 409/811] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:33.797052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057fce5-d09e-4bbb-9261-481ae17597f6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:33.843181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4c11c-23e1-4333-a632-121955c9c98d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:33.846292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:33.892154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd00c7f-9f74-44da-b7b7-b8e2cc59b5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:33.892766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00c7f-9f74-44da-b7b7-b8e2cc59b5fd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.893071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00c7f-9f74-44da-b7b7-b8e2cc59b5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.893596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd00c7f-9f74-44da-b7b7-b8e2cc59b5fd None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.894312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd00c7f-9f74-44da-b7b7-b8e2cc59b5fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:33.894981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd00c7f-9f74-44da-b7b7-b8e2cc59b5fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:33.896409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd00c7f-9f74-44da-b7b7-b8e2cc59b5fd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:33.911952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057fce5-d09e-4bbb-9261-481ae17597f6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:33.913138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 403/812] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:33.915213 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97730abf-d85f-4e0d-80e2-56417d2b2dbd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.916409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97730abf-d85f-4e0d-80e2-56417d2b2dbd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.916695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97730abf-d85f-4e0d-80e2-56417d2b2dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.939487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-97730abf-d85f-4e0d-80e2-56417d2b2dbd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:33.941247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-97730abf-d85f-4e0d-80e2-56417d2b2dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:33.950144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaaf718-addd-470f-bce3-2e9b16cdbb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:33.950895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaf718-addd-470f-bce3-2e9b16cdbb84 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:33.951266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaf718-addd-470f-bce3-2e9b16cdbb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:33.951678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaf718-addd-470f-bce3-2e9b16cdbb84 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:33.952393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaaf718-addd-470f-bce3-2e9b16cdbb84 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:33.952853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaaf718-addd-470f-bce3-2e9b16cdbb84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:33.954566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaaf718-addd-470f-bce3-2e9b16cdbb84 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:33.962031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 410/813] 10.222.0.84 () {56 vars in 1124 bytes} [Wed Aug 24 11:12:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:33.963059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaaf718-addd-470f-bce3-2e9b16cdbb84 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:34.004274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:34.017426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 404/814] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:34.018237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97730abf-d85f-4e0d-80e2-56417d2b2dbd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:34.080096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaaf718-addd-470f-bce3-2e9b16cdbb84 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:34.089418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:34.119237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e73d9f-4631-4bb9-a18a-badc52ca269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:34.120354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e73d9f-4631-4bb9-a18a-badc52ca269e None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:34.120867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e73d9f-4631-4bb9-a18a-badc52ca269e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:34.121372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e73d9f-4631-4bb9-a18a-badc52ca269e None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/1ba54c2312b348b08e0faae745747e7d/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:34.123543 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e73d9f-4631-4bb9-a18a-badc52ca269e None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=1ba54c2312b348b08e0faae745747e7d, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:34.125181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e73d9f-4631-4bb9-a18a-badc52ca269e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:34.145525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97730abf-d85f-4e0d-80e2-56417d2b2dbd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:34.181993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881662b-57f0-45e2-839a-183b9d2cc2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:34.182597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881662b-57f0-45e2-839a-183b9d2cc2aa None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:34.182869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881662b-57f0-45e2-839a-183b9d2cc2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:34.183136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881662b-57f0-45e2-839a-183b9d2cc2aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:34.183787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881662b-57f0-45e2-839a-183b9d2cc2aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:34.185244 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881662b-57f0-45e2-839a-183b9d2cc2aa None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:34.197293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 411/815] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:12:33 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/1ba54c2312b348b08e0faae745747e7d/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:34.198814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e73d9f-4631-4bb9-a18a-badc52ca269e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:34.283902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 405/816] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:12:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:34.310556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881662b-57f0-45e2-839a-183b9d2cc2aa None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:34.344383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e73d9f-4631-4bb9-a18a-badc52ca269e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:34.365434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:34.391828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe3817-92ea-4840-9976-7afb42967bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:34.392421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe3817-92ea-4840-9976-7afb42967bbe None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:34.392704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe3817-92ea-4840-9976-7afb42967bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:34.393001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe3817-92ea-4840-9976-7afb42967bbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:34.406308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe3817-92ea-4840-9976-7afb42967bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-87fe3817-92ea-4840-9976-7afb42967bbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:34.411837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:34.412791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:34.414298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 412/817] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:34.426018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe3817-92ea-4840-9976-7afb42967bbe None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:34.433730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881662b-57f0-45e2-839a-183b9d2cc2aa None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:34.468781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:34.477419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78682cbf-9266-4dc2-922a-5576ea755146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:34.478134 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682cbf-9266-4dc2-922a-5576ea755146 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:34.478414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682cbf-9266-4dc2-922a-5576ea755146 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:34.478690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78682cbf-9266-4dc2-922a-5576ea755146 None admin] PATH_INFO: `/v3/services/network` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:34.479286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78682cbf-9266-4dc2-922a-5576ea755146 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:34.480679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78682cbf-9266-4dc2-922a-5576ea755146 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-78682cbf-9266-4dc2-922a-5576ea755146 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:34.493491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:34.496074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:34.498207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 406/818] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:34 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:34.500992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78682cbf-9266-4dc2-922a-5576ea755146 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:34.634860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78682cbf-9266-4dc2-922a-5576ea755146 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:34.646401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:34.673502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe3817-92ea-4840-9976-7afb42967bbe None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:34.675802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d94b8-fc1e-49e9-849c-8bd3e1e8ddbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:34.676517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d94b8-fc1e-49e9-849c-8bd3e1e8ddbd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:34.676939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d94b8-fc1e-49e9-849c-8bd3e1e8ddbd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:34.677220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d94b8-fc1e-49e9-849c-8bd3e1e8ddbd None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:34.677862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d94b8-fc1e-49e9-849c-8bd3e1e8ddbd None admin] RBAC: Adding query filter params (name=network) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:34.678162 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d94b8-fc1e-49e9-849c-8bd3e1e8ddbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:34.679603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d94b8-fc1e-49e9-849c-8bd3e1e8ddbd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:34.685410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 407/819] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:34.691062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d94b8-fc1e-49e9-849c-8bd3e1e8ddbd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:34.712447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7a39f-4a0b-4b68-9393-b5705bf9d085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:34.713091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7a39f-4a0b-4b68-9393-b5705bf9d085 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:34.713367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7a39f-4a0b-4b68-9393-b5705bf9d085 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:34.713632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7a39f-4a0b-4b68-9393-b5705bf9d085 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:34.714290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7a39f-4a0b-4b68-9393-b5705bf9d085 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:34.714586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7a39f-4a0b-4b68-9393-b5705bf9d085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:34.715953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7a39f-4a0b-4b68-9393-b5705bf9d085 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:34.725533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 413/820] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:34.734739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7a39f-4a0b-4b68-9393-b5705bf9d085 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:34.811867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d94b8-fc1e-49e9-849c-8bd3e1e8ddbd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:34.857254 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889d25c-db87-443e-a5d0-37b37b16abad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:34.858128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889d25c-db87-443e-a5d0-37b37b16abad None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:34.858407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889d25c-db87-443e-a5d0-37b37b16abad None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:34.858675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889d25c-db87-443e-a5d0-37b37b16abad None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:34.859306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889d25c-db87-443e-a5d0-37b37b16abad None admin] RBAC: Adding query filter params (type=network) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:34.859603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889d25c-db87-443e-a5d0-37b37b16abad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:34.860996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889d25c-db87-443e-a5d0-37b37b16abad None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:34.868464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 408/821] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:34.876829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889d25c-db87-443e-a5d0-37b37b16abad None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:34.882358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:34.910405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7a39f-4a0b-4b68-9393-b5705bf9d085 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:34.951952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847699d-ce51-4da7-bca2-fcd6b652b28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:34.952855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847699d-ce51-4da7-bca2-fcd6b652b28e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:34.953278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847699d-ce51-4da7-bca2-fcd6b652b28e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:34.953718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847699d-ce51-4da7-bca2-fcd6b652b28e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:34.964647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847699d-ce51-4da7-bca2-fcd6b652b28e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:34.966385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847699d-ce51-4da7-bca2-fcd6b652b28e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-c847699d-ce51-4da7-bca2-fcd6b652b28e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:34.981833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:34.983329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:34.984435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:34.984435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:34.984435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:34.986203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 414/822] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:34.989191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:34.995488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847699d-ce51-4da7-bca2-fcd6b652b28e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:35.017661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889d25c-db87-443e-a5d0-37b37b16abad None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:35.085186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72825ae-9d2c-4fd6-a414-5d0ba437d9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:35.085859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72825ae-9d2c-4fd6-a414-5d0ba437d9de None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:35.086255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72825ae-9d2c-4fd6-a414-5d0ba437d9de None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:35.086531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72825ae-9d2c-4fd6-a414-5d0ba437d9de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:35.087241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72825ae-9d2c-4fd6-a414-5d0ba437d9de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:35.088617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72825ae-9d2c-4fd6-a414-5d0ba437d9de None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:35.151024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847699d-ce51-4da7-bca2-fcd6b652b28e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:35.188999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5c289-6bd3-464d-8faf-89cfd6de2575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:35.189837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5c289-6bd3-464d-8faf-89cfd6de2575 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:35.190264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5c289-6bd3-464d-8faf-89cfd6de2575 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:35.190699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5c289-6bd3-464d-8faf-89cfd6de2575 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:35.191481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5c289-6bd3-464d-8faf-89cfd6de2575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:35.191926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5c289-6bd3-464d-8faf-89cfd6de2575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:35.193519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5c289-6bd3-464d-8faf-89cfd6de2575 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:35.206767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 415/823] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:35.207365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:35.209647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 409/824] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:12:34 2022] POST /identity/v3/endpoints => generated 337 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:35.215463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5c289-6bd3-464d-8faf-89cfd6de2575 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:35.328137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5c289-6bd3-464d-8faf-89cfd6de2575 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:35.373502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca1f30-e7e2-4f7b-8c06-a39a32e0a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:35.374392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca1f30-e7e2-4f7b-8c06-a39a32e0a228 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:35.374852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca1f30-e7e2-4f7b-8c06-a39a32e0a228 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:35.375275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca1f30-e7e2-4f7b-8c06-a39a32e0a228 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:35.383549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca1f30-e7e2-4f7b-8c06-a39a32e0a228 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:35.385361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca1f30-e7e2-4f7b-8c06-a39a32e0a228 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-99ca1f30-e7e2-4f7b-8c06-a39a32e0a228 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:35.398215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:35.400489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:12:35.401702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:35.403943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 416/825] 10.222.0.84 () {56 vars in 1093 bytes} [Wed Aug 24 11:12:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:35.409369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72825ae-9d2c-4fd6-a414-5d0ba437d9de None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:35.514553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:35.524810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72825ae-9d2c-4fd6-a414-5d0ba437d9de None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:35.561161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ef8e7-9326-4e84-ad62-f0f579e8b5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:35.561774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ef8e7-9326-4e84-ad62-f0f579e8b5be None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:35.562111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ef8e7-9326-4e84-ad62-f0f579e8b5be None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:35.562387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ef8e7-9326-4e84-ad62-f0f579e8b5be None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:35.563111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ef8e7-9326-4e84-ad62-f0f579e8b5be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:35.563417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ef8e7-9326-4e84-ad62-f0f579e8b5be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:35.564867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ef8e7-9326-4e84-ad62-f0f579e8b5be None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:35.583459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 410/826] 10.222.0.84 () {56 vars in 1133 bytes} [Wed Aug 24 11:12:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:35.589142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca1f30-e7e2-4f7b-8c06-a39a32e0a228 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:35.646931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:35.695209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca1f30-e7e2-4f7b-8c06-a39a32e0a228 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:35.733195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181cdb4-916f-4baf-b3f4-143858f172e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:35.733830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cdb4-916f-4baf-b3f4-143858f172e9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:35.734106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cdb4-916f-4baf-b3f4-143858f172e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:35.734371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cdb4-916f-4baf-b3f4-143858f172e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:35.808964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181cdb4-916f-4baf-b3f4-143858f172e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:35.810446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181cdb4-916f-4baf-b3f4-143858f172e9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:35.863218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:35.911606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e0504-75bf-4355-882e-bd053017fa01 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:35.912035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e0504-75bf-4355-882e-bd053017fa01 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:35.912414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e0504-75bf-4355-882e-bd053017fa01 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:35.914667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 411/827] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:35.914981 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 417/828] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:12:35.919535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e0504-75bf-4355-882e-bd053017fa01 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:35.920167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8a313-ff0a-4e49-82f3-edbf55dc012f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:35.920454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8a313-ff0a-4e49-82f3-edbf55dc012f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:35.920730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8a313-ff0a-4e49-82f3-edbf55dc012f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:35.940557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-8fc8a313-ff0a-4e49-82f3-edbf55dc012f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:35.941814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8fc8a313-ff0a-4e49-82f3-edbf55dc012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:36.021822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 418/829] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:36.049162 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e0504-75bf-4355-882e-bd053017fa01 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:36.086358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e7427-700e-4d61-a692-820a8d9484fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:36.086970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e7427-700e-4d61-a692-820a8d9484fa None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.087253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e7427-700e-4d61-a692-820a8d9484fa None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.087525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e7427-700e-4d61-a692-820a8d9484fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.098225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e7427-700e-4d61-a692-820a8d9484fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:36.099694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e7427-700e-4d61-a692-820a8d9484fa None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e69e7427-700e-4d61-a692-820a8d9484fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:36.115634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:36.118168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:36.119253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:36.119253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:36.119253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:36.119253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 412/830] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:36.125901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8a313-ff0a-4e49-82f3-edbf55dc012f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:36.215043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:36.234202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8a313-ff0a-4e49-82f3-edbf55dc012f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:36.268856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6f338-f8dc-425a-8a10-f0cd908135af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:36.269463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6f338-f8dc-425a-8a10-f0cd908135af None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.269744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6f338-f8dc-425a-8a10-f0cd908135af None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.270009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6f338-f8dc-425a-8a10-f0cd908135af None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.270642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6f338-f8dc-425a-8a10-f0cd908135af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:36.270935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6f338-f8dc-425a-8a10-f0cd908135af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:36.272435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad6f338-f8dc-425a-8a10-f0cd908135af None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:36.285399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 419/831] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:36.291578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e7427-700e-4d61-a692-820a8d9484fa None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:36.294726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:36.300714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6f338-f8dc-425a-8a10-f0cd908135af None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:36.416062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e7427-700e-4d61-a692-820a8d9484fa None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:36.429095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6f338-f8dc-425a-8a10-f0cd908135af None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:36.459568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4926b-81ab-488c-9402-2216c5da4bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:36.460316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4926b-81ab-488c-9402-2216c5da4bbd None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.461815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4926b-81ab-488c-9402-2216c5da4bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.462308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4926b-81ab-488c-9402-2216c5da4bbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.471051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4926b-81ab-488c-9402-2216c5da4bbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:36.475387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db91edb-b1f2-4e3a-9ca7-38dfd82b5ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:36.475523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4926b-81ab-488c-9402-2216c5da4bbd None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:36.475615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db91edb-b1f2-4e3a-9ca7-38dfd82b5ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.475615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db91edb-b1f2-4e3a-9ca7-38dfd82b5ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.475615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db91edb-b1f2-4e3a-9ca7-38dfd82b5ed0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-b8b4926b-81ab-488c-9402-2216c5da4bbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:36.486747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:36.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:36.488568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:36.488568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:36.488568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:36.488568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:36.488568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:36.488568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:36.488568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:36.488568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:36.489612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db91edb-b1f2-4e3a-9ca7-38dfd82b5ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:36.492829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 413/832] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:36.493745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4926b-81ab-488c-9402-2216c5da4bbd None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-3db91edb-b1f2-4e3a-9ca7-38dfd82b5ed0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:36.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:36.496576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:12:36.497438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:36.499993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 420/833] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:12:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:36.504733 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db91edb-b1f2-4e3a-9ca7-38dfd82b5ed0 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:36.508636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:36.576768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:36.624057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4926b-81ab-488c-9402-2216c5da4bbd None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:36.629649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db91edb-b1f2-4e3a-9ca7-38dfd82b5ed0 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:36.667095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24713a-eb70-4128-b0eb-5b58f6770298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:36.667344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c4436-1649-4b5d-9265-f370501ed6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:36.667625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24713a-eb70-4128-b0eb-5b58f6770298 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.667903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24713a-eb70-4128-b0eb-5b58f6770298 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.668173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24713a-eb70-4128-b0eb-5b58f6770298 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.668572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c4436-1649-4b5d-9265-f370501ed6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.668887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24713a-eb70-4128-b0eb-5b58f6770298 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:36.669183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c4436-1649-4b5d-9265-f370501ed6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.669304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24713a-eb70-4128-b0eb-5b58f6770298 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:36.669694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c4436-1649-4b5d-9265-f370501ed6d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.670486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c4436-1649-4b5d-9265-f370501ed6d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:36.670596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24713a-eb70-4128-b0eb-5b58f6770298 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:36.671019 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c4436-1649-4b5d-9265-f370501ed6d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:36.672499 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c4436-1649-4b5d-9265-f370501ed6d3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:36.686401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 421/834] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:12:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:36.686401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811ec6f-48d0-4398-84d1-5ba97d488fb0 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.686401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811ec6f-48d0-4398-84d1-5ba97d488fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.686401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811ec6f-48d0-4398-84d1-5ba97d488fb0 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.686401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 422/835] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:36.687288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811ec6f-48d0-4398-84d1-5ba97d488fb0 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:36.689155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 414/836] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:36.690507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24713a-eb70-4128-b0eb-5b58f6770298 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:36.749918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:36.787224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811ec6f-48d0-4398-84d1-5ba97d488fb0 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:36.789955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24713a-eb70-4128-b0eb-5b58f6770298 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:36.812854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:36.821372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51262bb-266f-4a06-98d3-1ec43afc9b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:36.821995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51262bb-266f-4a06-98d3-1ec43afc9b51 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.822271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51262bb-266f-4a06-98d3-1ec43afc9b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.822540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51262bb-266f-4a06-98d3-1ec43afc9b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.824488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44f48f-d8da-4085-8bf7-b61e088bc844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:36.824658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44f48f-d8da-4085-8bf7-b61e088bc844 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.825901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44f48f-d8da-4085-8bf7-b61e088bc844 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.825901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44f48f-d8da-4085-8bf7-b61e088bc844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.826876 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44f48f-d8da-4085-8bf7-b61e088bc844 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=f58a559951c94d68951da78c82341a00) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:36.827184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44f48f-d8da-4085-8bf7-b61e088bc844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:36.828599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc44f48f-d8da-4085-8bf7-b61e088bc844 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:36.834102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51262bb-266f-4a06-98d3-1ec43afc9b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:36.840538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 415/837] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:36 2022] GET /identity/v3/role_assignments?user.id=f58a559951c94d68951da78c82341a00&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:36.840538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313810a-31fb-448d-91a0-b3a034d3cd6e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.840853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51262bb-266f-4a06-98d3-1ec43afc9b51 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:36.840947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313810a-31fb-448d-91a0-b3a034d3cd6e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.840947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313810a-31fb-448d-91a0-b3a034d3cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.858761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0313810a-31fb-448d-91a0-b3a034d3cd6e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:36.859985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0313810a-31fb-448d-91a0-b3a034d3cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-f51262bb-266f-4a06-98d3-1ec43afc9b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:36.863441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:36.864433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:36.865363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:36.865363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:36.865363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:36.865363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 423/838] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:36.865933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00c25b-2b07-490d-8a7c-bb824262d327 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.866218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00c25b-2b07-490d-8a7c-bb824262d327 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.866487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00c25b-2b07-490d-8a7c-bb824262d327 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.873595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 424/839] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:36.874598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00c25b-2b07-490d-8a7c-bb824262d327 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:36.940987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 416/840] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:36.942616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef9c4-17d9-4800-b42b-410d32f62b08 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:36.942900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef9c4-17d9-4800-b42b-410d32f62b08 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:36.943164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef9c4-17d9-4800-b42b-410d32f62b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:36.962229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-993ef9c4-17d9-4800-b42b-410d32f62b08 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:36.963529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-993ef9c4-17d9-4800-b42b-410d32f62b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:37.016770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00c25b-2b07-490d-8a7c-bb824262d327 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:37.034846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:37.048730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 417/841] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:37.057589 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b906f8b-a998-4afb-8a26-0aad699e6967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:37.057740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b906f8b-a998-4afb-8a26-0aad699e6967 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:37.057967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b906f8b-a998-4afb-8a26-0aad699e6967 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:37.058228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b906f8b-a998-4afb-8a26-0aad699e6967 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:37.058916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b906f8b-a998-4afb-8a26-0aad699e6967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:37.059272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b906f8b-a998-4afb-8a26-0aad699e6967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:37.060617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b906f8b-a998-4afb-8a26-0aad699e6967 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:37.076013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 425/842] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:37.082946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ef9c4-17d9-4800-b42b-410d32f62b08 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:37.132075 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:37.195470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ef9c4-17d9-4800-b42b-410d32f62b08 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:37.217176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b906f8b-a998-4afb-8a26-0aad699e6967 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:37.232970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4fbe-af42-48fb-bb4e-38cb3f84197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:37.233603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4fbe-af42-48fb-bb4e-38cb3f84197e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:37.233877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4fbe-af42-48fb-bb4e-38cb3f84197e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:37.234147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4fbe-af42-48fb-bb4e-38cb3f84197e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:37.241760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb4fbe-af42-48fb-bb4e-38cb3f84197e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:37.243564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb4fbe-af42-48fb-bb4e-38cb3f84197e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-dbbb4fbe-af42-48fb-bb4e-38cb3f84197e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:37.256297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:37.257278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:37.258830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 418/843] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:37.265873 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4fbe-af42-48fb-bb4e-38cb3f84197e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:37.325184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b906f8b-a998-4afb-8a26-0aad699e6967 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:37.361729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84598839-66c0-48c0-99d5-aa3e34066a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:37.362470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84598839-66c0-48c0-99d5-aa3e34066a22 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:37.362746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84598839-66c0-48c0-99d5-aa3e34066a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:37.363011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84598839-66c0-48c0-99d5-aa3e34066a22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:37.373928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84598839-66c0-48c0-99d5-aa3e34066a22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-84598839-66c0-48c0-99d5-aa3e34066a22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:37.378964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:37.379874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:37.380720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 426/844] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:37.381461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84598839-66c0-48c0-99d5-aa3e34066a22 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:37.396983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4fbe-af42-48fb-bb4e-38cb3f84197e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:37.407196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:37.422116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:37.436663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f152b17-0184-4018-b714-46c8d4fee17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:37.437431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f152b17-0184-4018-b714-46c8d4fee17f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:37.437768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f152b17-0184-4018-b714-46c8d4fee17f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:37.438040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f152b17-0184-4018-b714-46c8d4fee17f None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:37.438759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f152b17-0184-4018-b714-46c8d4fee17f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:37.439057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f152b17-0184-4018-b714-46c8d4fee17f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:37.440528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f152b17-0184-4018-b714-46c8d4fee17f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:37.464250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 419/845] 10.222.0.84 () {56 vars in 1149 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:37.465018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f152b17-0184-4018-b714-46c8d4fee17f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:37.488631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84598839-66c0-48c0-99d5-aa3e34066a22 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:37.525432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76085-23c2-4c5c-bad5-5edae9da7c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:37.526062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76085-23c2-4c5c-bad5-5edae9da7c97 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:37.526339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76085-23c2-4c5c-bad5-5edae9da7c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:37.526952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76085-23c2-4c5c-bad5-5edae9da7c97 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:37.527275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b76085-23c2-4c5c-bad5-5edae9da7c97 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:37.528729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b76085-23c2-4c5c-bad5-5edae9da7c97 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-f0b76085-23c2-4c5c-bad5-5edae9da7c97 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:37.540528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:37.541734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:37.542705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 427/846] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:37.543673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76085-23c2-4c5c-bad5-5edae9da7c97 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:37.574403 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f152b17-0184-4018-b714-46c8d4fee17f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:37.609511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfac59f-c3d4-48d1-92ea-6495727f4a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:37.610165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfac59f-c3d4-48d1-92ea-6495727f4a34 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:37.610442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfac59f-c3d4-48d1-92ea-6495727f4a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:37.610708 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfac59f-c3d4-48d1-92ea-6495727f4a34 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:37.611388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfac59f-c3d4-48d1-92ea-6495727f4a34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:37.611692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfac59f-c3d4-48d1-92ea-6495727f4a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:37.613070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfac59f-c3d4-48d1-92ea-6495727f4a34 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:37.626372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 420/847] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:37.626372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfac59f-c3d4-48d1-92ea-6495727f4a34 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:37.656443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:37.661433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:37.665478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76085-23c2-4c5c-bad5-5edae9da7c97 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:37.699879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9f3e0-3aab-497f-9382-a2dd3d46ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:37.700479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9f3e0-3aab-497f-9382-a2dd3d46ffb5 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:37.700756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9f3e0-3aab-497f-9382-a2dd3d46ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:37.701054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9f3e0-3aab-497f-9382-a2dd3d46ffb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:37.737145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfac59f-c3d4-48d1-92ea-6495727f4a34 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:37.778514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ba2b2-d184-45f5-a1f1-0e2922d3e2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:37.779466 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ba2b2-d184-45f5-a1f1-0e2922d3e2ed None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:37.779741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ba2b2-d184-45f5-a1f1-0e2922d3e2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:37.780011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ba2b2-d184-45f5-a1f1-0e2922d3e2ed None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:37.780637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ba2b2-d184-45f5-a1f1-0e2922d3e2ed None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:37.780970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ba2b2-d184-45f5-a1f1-0e2922d3e2ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:37.781898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9f3e0-3aab-497f-9382-a2dd3d46ffb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:37.782413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ba2b2-d184-45f5-a1f1-0e2922d3e2ed None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:37.784245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9f3e0-3aab-497f-9382-a2dd3d46ffb5 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:37.792718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 421/848] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:37.792718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ba2b2-d184-45f5-a1f1-0e2922d3e2ed None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:37.887317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:37.898790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ba2b2-d184-45f5-a1f1-0e2922d3e2ed None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:37.903376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:37.907003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 428/849] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:12:37.911706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9f3e0-3aab-497f-9382-a2dd3d46ffb5 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:37.935746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2760d-01b4-401e-a3db-fea2fd458379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:37.936375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2760d-01b4-401e-a3db-fea2fd458379 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:37.936657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2760d-01b4-401e-a3db-fea2fd458379 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:37.936953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2760d-01b4-401e-a3db-fea2fd458379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:37.947740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2760d-01b4-401e-a3db-fea2fd458379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:37.949214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb2760d-01b4-401e-a3db-fea2fd458379 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-cdb2760d-01b4-401e-a3db-fea2fd458379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:37.964076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:37.965122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:37.965986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:37.965986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:37.965986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:37.965986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 422/850] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:37.967114 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2760d-01b4-401e-a3db-fea2fd458379 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:38.025987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9f3e0-3aab-497f-9382-a2dd3d46ffb5 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.064369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f287f-7716-43ea-a055-b1f16b3f3d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.065341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f287f-7716-43ea-a055-b1f16b3f3d77 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.065808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f287f-7716-43ea-a055-b1f16b3f3d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.066075 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f287f-7716-43ea-a055-b1f16b3f3d77 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.066691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f287f-7716-43ea-a055-b1f16b3f3d77 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:38.066989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f287f-7716-43ea-a055-b1f16b3f3d77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.068369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f287f-7716-43ea-a055-b1f16b3f3d77 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.073477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 429/851] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:38.074272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f287f-7716-43ea-a055-b1f16b3f3d77 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:38.089147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2760d-01b4-401e-a3db-fea2fd458379 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.130898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e823a-2d53-4796-b5b9-78cf8c2729f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.131987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e823a-2d53-4796-b5b9-78cf8c2729f8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.132498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e823a-2d53-4796-b5b9-78cf8c2729f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.133454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e823a-2d53-4796-b5b9-78cf8c2729f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.147329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e823a-2d53-4796-b5b9-78cf8c2729f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.149619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e823a-2d53-4796-b5b9-78cf8c2729f8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.152040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-2b1e823a-2d53-4796-b5b9-78cf8c2729f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:38.167338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:38.168414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:38.169408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:38.169408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:38.169408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:38.169408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 423/852] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:38.174342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e823a-2d53-4796-b5b9-78cf8c2729f8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:38.185943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:38.229051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f287f-7716-43ea-a055-b1f16b3f3d77 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.270927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520fdf8-8ea4-43b6-8604-2331dca5c28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.272291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520fdf8-8ea4-43b6-8604-2331dca5c28a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.272567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520fdf8-8ea4-43b6-8604-2331dca5c28a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.273161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520fdf8-8ea4-43b6-8604-2331dca5c28a None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.274367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520fdf8-8ea4-43b6-8604-2331dca5c28a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:38.274669 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520fdf8-8ea4-43b6-8604-2331dca5c28a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.277246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520fdf8-8ea4-43b6-8604-2331dca5c28a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.292732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 430/853] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:38.298258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520fdf8-8ea4-43b6-8604-2331dca5c28a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:38.323047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e823a-2d53-4796-b5b9-78cf8c2729f8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.371785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6be0da-c41a-4986-aada-5135f963b2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.372394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6be0da-c41a-4986-aada-5135f963b2cf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.372667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6be0da-c41a-4986-aada-5135f963b2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.373223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6be0da-c41a-4986-aada-5135f963b2cf None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.374731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6be0da-c41a-4986-aada-5135f963b2cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:38.375051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6be0da-c41a-4986-aada-5135f963b2cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.376425 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6be0da-c41a-4986-aada-5135f963b2cf None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.396880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 424/854] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:38.398103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6be0da-c41a-4986-aada-5135f963b2cf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:38.411981 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520fdf8-8ea4-43b6-8604-2331dca5c28a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.439300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:38.451994 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59200f9e-ab8e-4f64-a4e2-758440d058ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.463384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59200f9e-ab8e-4f64-a4e2-758440d058ef None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.463384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59200f9e-ab8e-4f64-a4e2-758440d058ef None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.463384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59200f9e-ab8e-4f64-a4e2-758440d058ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.465098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59200f9e-ab8e-4f64-a4e2-758440d058ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.466566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59200f9e-ab8e-4f64-a4e2-758440d058ef None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-59200f9e-ab8e-4f64-a4e2-758440d058ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:38.478523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:38.479706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 11:12:38.480713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:38.481776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:38.484590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 431/855] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:12:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:38.485906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59200f9e-ab8e-4f64-a4e2-758440d058ef None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:38.536876 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6be0da-c41a-4986-aada-5135f963b2cf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.575786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589e67b-6461-4040-a883-9a3dc5a5768b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.576788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589e67b-6461-4040-a883-9a3dc5a5768b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.577239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589e67b-6461-4040-a883-9a3dc5a5768b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.577832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589e67b-6461-4040-a883-9a3dc5a5768b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.585454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589e67b-6461-4040-a883-9a3dc5a5768b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.587679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589e67b-6461-4040-a883-9a3dc5a5768b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.595319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59200f9e-ab8e-4f64-a4e2-758440d058ef None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-9589e67b-6461-4040-a883-9a3dc5a5768b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:38.600000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:38.601080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:38.602890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:38.602890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:38.602890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:38.602890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:38.602890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:38.602890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:38.602890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:38.602890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:38.609773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 425/856] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:38.610815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589e67b-6461-4040-a883-9a3dc5a5768b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:38.634630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca6b71-cf6b-4cbc-84f8-bde5315a5d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.635420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca6b71-cf6b-4cbc-84f8-bde5315a5d53 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.635765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca6b71-cf6b-4cbc-84f8-bde5315a5d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.636176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca6b71-cf6b-4cbc-84f8-bde5315a5d53 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.637041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca6b71-cf6b-4cbc-84f8-bde5315a5d53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:38.637464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca6b71-cf6b-4cbc-84f8-bde5315a5d53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.638992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca6b71-cf6b-4cbc-84f8-bde5315a5d53 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.669337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 432/857] 10.222.0.84 () {56 vars in 1135 bytes} [Wed Aug 24 11:12:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:38.670049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca6b71-cf6b-4cbc-84f8-bde5315a5d53 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:38.726812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:38.754056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589e67b-6461-4040-a883-9a3dc5a5768b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.772942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:38.800103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1cea6-b5e0-4add-98f4-ce60e033b695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.800812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1cea6-b5e0-4add-98f4-ce60e033b695 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.801199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1cea6-b5e0-4add-98f4-ce60e033b695 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.801485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca6b71-cf6b-4cbc-84f8-bde5315a5d53 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.801645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1cea6-b5e0-4add-98f4-ce60e033b695 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.802450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1cea6-b5e0-4add-98f4-ce60e033b695 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:38.803113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1cea6-b5e0-4add-98f4-ce60e033b695 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.806923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1cea6-b5e0-4add-98f4-ce60e033b695 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.823005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 426/858] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:38.829034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1cea6-b5e0-4add-98f4-ce60e033b695 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:38.850794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6bdb9-90d2-472f-bc69-e441f565e271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.852198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6bdb9-90d2-472f-bc69-e441f565e271 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.852775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6bdb9-90d2-472f-bc69-e441f565e271 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.853172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6bdb9-90d2-472f-bc69-e441f565e271 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.928059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6bdb9-90d2-472f-bc69-e441f565e271 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.930396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6bdb9-90d2-472f-bc69-e441f565e271 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.937621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1cea6-b5e0-4add-98f4-ce60e033b695 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:38.974522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:38.975465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d981ec7-788c-48cc-a408-310f4af34585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:38.976171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d981ec7-788c-48cc-a408-310f4af34585 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.976923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d981ec7-788c-48cc-a408-310f4af34585 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.977154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d981ec7-788c-48cc-a408-310f4af34585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.977871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d981ec7-788c-48cc-a408-310f4af34585 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=164f3bcc6363445da191bd1b78499a53) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:38.978170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d981ec7-788c-48cc-a408-310f4af34585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:38.979707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d981ec7-788c-48cc-a408-310f4af34585 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:38.996176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 427/859] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:38 2022] GET /identity/v3/role_assignments?user.id=164f3bcc6363445da191bd1b78499a53&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:38.996176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432cad4-089b-46eb-a1e0-d47f4aaf7ee3 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.996176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432cad4-089b-46eb-a1e0-d47f4aaf7ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.996176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432cad4-089b-46eb-a1e0-d47f4aaf7ee3 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:38.996176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 428/860] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:38.997533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8c9c1-2fc8-46d7-a6c7-d1d7245585c5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:38.997810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8c9c1-2fc8-46d7-a6c7-d1d7245585c5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:38.998079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8c9c1-2fc8-46d7-a6c7-d1d7245585c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.023437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-77e8c9c1-2fc8-46d7-a6c7-d1d7245585c5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:39.025201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-77e8c9c1-2fc8-46d7-a6c7-d1d7245585c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:39.060578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:39.061416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 433/861] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:12:39.067754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6bdb9-90d2-472f-bc69-e441f565e271 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:39.113309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 429/862] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:39.182498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6bdb9-90d2-472f-bc69-e441f565e271 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:39.219996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b6812-e809-45e3-ab9b-0d14dee039f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:39.221046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b6812-e809-45e3-ab9b-0d14dee039f1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:39.221473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b6812-e809-45e3-ab9b-0d14dee039f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:39.221907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b6812-e809-45e3-ab9b-0d14dee039f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.234481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b6812-e809-45e3-ab9b-0d14dee039f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:39.236599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b6812-e809-45e3-ab9b-0d14dee039f1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:39.238721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-cd6b6812-e809-45e3-ab9b-0d14dee039f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:39.252066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:39.255051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:39.256013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:39.256013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:39.256013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:39.256013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 434/863] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:39.258877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8c9c1-2fc8-46d7-a6c7-d1d7245585c5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:39.364228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8c9c1-2fc8-46d7-a6c7-d1d7245585c5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:39.386322 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b6812-e809-45e3-ab9b-0d14dee039f1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:39.404442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53e441-f927-40c0-b9da-ea2113470d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:39.405827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53e441-f927-40c0-b9da-ea2113470d08 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:39.406583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53e441-f927-40c0-b9da-ea2113470d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:39.407073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53e441-f927-40c0-b9da-ea2113470d08 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.408290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53e441-f927-40c0-b9da-ea2113470d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:39.408821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53e441-f927-40c0-b9da-ea2113470d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:39.410922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53e441-f927-40c0-b9da-ea2113470d08 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:39.427257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 430/864] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:39.433565 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53e441-f927-40c0-b9da-ea2113470d08 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:39.464474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:39.512290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b6812-e809-45e3-ab9b-0d14dee039f1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:39.551154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b674b9-11d0-4cd2-b83b-c1709f6de762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:39.552034 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b674b9-11d0-4cd2-b83b-c1709f6de762 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:39.552485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b674b9-11d0-4cd2-b83b-c1709f6de762 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:39.552928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b674b9-11d0-4cd2-b83b-c1709f6de762 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.554185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b674b9-11d0-4cd2-b83b-c1709f6de762 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:39.556827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b674b9-11d0-4cd2-b83b-c1709f6de762 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:39.567252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53e441-f927-40c0-b9da-ea2113470d08 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-41b674b9-11d0-4cd2-b83b-c1709f6de762 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:39.571120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:39.572251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:12:39.573445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:39.575854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 435/865] 10.222.0.84 () {56 vars in 1132 bytes} [Wed Aug 24 11:12:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:39.582121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b674b9-11d0-4cd2-b83b-c1709f6de762 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:39.614495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e4705-cacf-452b-9dce-aeb312fe3352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:39.615133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4705-cacf-452b-9dce-aeb312fe3352 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:39.615478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4705-cacf-452b-9dce-aeb312fe3352 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:39.615949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4705-cacf-452b-9dce-aeb312fe3352 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.625834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e4705-cacf-452b-9dce-aeb312fe3352 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:39.627940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e4705-cacf-452b-9dce-aeb312fe3352 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-eb3e4705-cacf-452b-9dce-aeb312fe3352 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:39.642554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:39.643520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:39.644388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:39.644388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:39.644388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:39.644388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:39.644388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:39.644388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:39.644388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:39.644388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:39.644388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 431/866] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:39.650502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e4705-cacf-452b-9dce-aeb312fe3352 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:39.659565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:39.703448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b674b9-11d0-4cd2-b83b-c1709f6de762 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:39.739424 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518c129-5b91-4d94-9291-2e3b83c424dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:39.740042 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518c129-5b91-4d94-9291-2e3b83c424dc None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:39.740299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518c129-5b91-4d94-9291-2e3b83c424dc None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:39.740562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518c129-5b91-4d94-9291-2e3b83c424dc None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.741281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518c129-5b91-4d94-9291-2e3b83c424dc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:39.741575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518c129-5b91-4d94-9291-2e3b83c424dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:39.743528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518c129-5b91-4d94-9291-2e3b83c424dc None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:39.758108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 436/867] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:39.763120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518c129-5b91-4d94-9291-2e3b83c424dc None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:39.777620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:39.793178 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e4705-cacf-452b-9dce-aeb312fe3352 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:39.837977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913f361-dd21-4795-b4b5-87364baf4f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:39.839000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913f361-dd21-4795-b4b5-87364baf4f9b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:39.839480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913f361-dd21-4795-b4b5-87364baf4f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:39.839998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913f361-dd21-4795-b4b5-87364baf4f9b None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.841025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913f361-dd21-4795-b4b5-87364baf4f9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:39.841632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913f361-dd21-4795-b4b5-87364baf4f9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:39.843342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913f361-dd21-4795-b4b5-87364baf4f9b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:39.857145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 432/868] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:39.867671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913f361-dd21-4795-b4b5-87364baf4f9b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:39.884849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518c129-5b91-4d94-9291-2e3b83c424dc None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:39.907635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:39.920594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4df533-9b88-4bce-8737-ce3c1e44473e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:39.921404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4df533-9b88-4bce-8737-ce3c1e44473e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:39.921746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4df533-9b88-4bce-8737-ce3c1e44473e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:39.922083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4df533-9b88-4bce-8737-ce3c1e44473e None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:39.922768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4df533-9b88-4bce-8737-ce3c1e44473e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:39.923137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4df533-9b88-4bce-8737-ce3c1e44473e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:39.924723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4df533-9b88-4bce-8737-ce3c1e44473e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:39.930831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 437/869] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:39.978938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913f361-dd21-4795-b4b5-87364baf4f9b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:40.018737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8cc843-3cb0-4c49-afe4-ef5c7e804b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:40.019356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cc843-3cb0-4c49-afe4-ef5c7e804b46 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:40.019635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cc843-3cb0-4c49-afe4-ef5c7e804b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:40.019907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8cc843-3cb0-4c49-afe4-ef5c7e804b46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:40.020655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cc843-3cb0-4c49-afe4-ef5c7e804b46 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=1ba54c2312b348b08e0faae745747e7d) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:40.020995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cc843-3cb0-4c49-afe4-ef5c7e804b46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:40.022464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8cc843-3cb0-4c49-afe4-ef5c7e804b46 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:40.030235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 433/870] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:39 2022] GET /identity/v3/role_assignments?user.id=1ba54c2312b348b08e0faae745747e7d&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:40.065787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4f98-c11f-42d8-b1ef-3a12fda63cae None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:40.066216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4f98-c11f-42d8-b1ef-3a12fda63cae None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:40.066684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4f98-c11f-42d8-b1ef-3a12fda63cae None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:40.068737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 438/871] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:40.074285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e5e68-0087-493f-b6fb-aa626717da65 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:40.074632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e5e68-0087-493f-b6fb-aa626717da65 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:40.074808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e5e68-0087-493f-b6fb-aa626717da65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:40.095096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e22e5e68-0087-493f-b6fb-aa626717da65 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:40.096314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e22e5e68-0087-493f-b6fb-aa626717da65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:40.106342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:40.180952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 434/872] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:40.451787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad4f98-c11f-42d8-b1ef-3a12fda63cae None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:40.562969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad4f98-c11f-42d8-b1ef-3a12fda63cae None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:40.597703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99524ad9-6bce-4cc6-809f-927cf0760648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:40.598304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99524ad9-6bce-4cc6-809f-927cf0760648 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:40.598575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99524ad9-6bce-4cc6-809f-927cf0760648 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:40.598834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99524ad9-6bce-4cc6-809f-927cf0760648 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:40.599475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99524ad9-6bce-4cc6-809f-927cf0760648 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:40.600918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99524ad9-6bce-4cc6-809f-927cf0760648 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:40.690135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 439/873] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:12:40 2022] POST /identity/v3/services => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:41.151691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271663a-08d7-4726-bd23-36e49f4e972f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:41.152726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271663a-08d7-4726-bd23-36e49f4e972f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:41.153252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271663a-08d7-4726-bd23-36e49f4e972f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:41.155056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 435/874] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:41.160596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61e8e6-f55e-4ceb-bbae-cab8baf0a9fc None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:41.161158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61e8e6-f55e-4ceb-bbae-cab8baf0a9fc None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:41.161737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61e8e6-f55e-4ceb-bbae-cab8baf0a9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:41.181553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-5f61e8e6-f55e-4ceb-bbae-cab8baf0a9fc None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:41.183144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5f61e8e6-f55e-4ceb-bbae-cab8baf0a9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:41.234582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:41.270562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 440/875] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:41.544113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271663a-08d7-4726-bd23-36e49f4e972f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:41.667054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271663a-08d7-4726-bd23-36e49f4e972f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:41.704506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaacd8-8fca-4606-b49e-33d9f827c29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:41.705172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaacd8-8fca-4606-b49e-33d9f827c29e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:41.705451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaacd8-8fca-4606-b49e-33d9f827c29e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:41.705726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaacd8-8fca-4606-b49e-33d9f827c29e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:41.717260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eaacd8-8fca-4606-b49e-33d9f827c29e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f8eaacd8-8fca-4606-b49e-33d9f827c29e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:41.722113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:41.723014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:41.724223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 436/876] 10.222.0.84 () {56 vars in 1120 bytes} [Wed Aug 24 11:12:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:41.729905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61e8e6-f55e-4ceb-bbae-cab8baf0a9fc None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:41.840725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61e8e6-f55e-4ceb-bbae-cab8baf0a9fc None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:41.877692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db9666-b0e6-42e5-94fa-36be57bc81e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:41.878295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db9666-b0e6-42e5-94fa-36be57bc81e8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:41.878572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db9666-b0e6-42e5-94fa-36be57bc81e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:41.878841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db9666-b0e6-42e5-94fa-36be57bc81e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:41.879503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db9666-b0e6-42e5-94fa-36be57bc81e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:41.879802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db9666-b0e6-42e5-94fa-36be57bc81e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:41.881334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db9666-b0e6-42e5-94fa-36be57bc81e8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:41.890557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 441/877] 10.222.0.84 () {56 vars in 1115 bytes} [Wed Aug 24 11:12:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:41.896189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaacd8-8fca-4606-b49e-33d9f827c29e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:41.900220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:42.005966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378459-79c7-4491-9a2c-fed385de004a None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.006320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378459-79c7-4491-9a2c-fed385de004a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.006513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378459-79c7-4491-9a2c-fed385de004a None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.007245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaacd8-8fca-4606-b49e-33d9f827c29e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:42.008446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 442/878] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:12:42.013547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209280f5-32ce-48b5-adde-cc0bbca61f3f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.013828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209280f5-32ce-48b5-adde-cc0bbca61f3f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.014105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209280f5-32ce-48b5-adde-cc0bbca61f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.033403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-209280f5-32ce-48b5-adde-cc0bbca61f3f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:42.034637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-209280f5-32ce-48b5-adde-cc0bbca61f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:42.051272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49064-388f-4144-b3a2-c28cdfded822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:42.052037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49064-388f-4144-b3a2-c28cdfded822 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.052315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49064-388f-4144-b3a2-c28cdfded822 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.052581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49064-388f-4144-b3a2-c28cdfded822 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.063579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49064-388f-4144-b3a2-c28cdfded822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:42.065050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49064-388f-4144-b3a2-c28cdfded822 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:42.065389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5de49064-388f-4144-b3a2-c28cdfded822 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:42.079698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:42.080958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:42.081878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:42.081878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:42.081878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:42.081878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 437/879] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:42.082103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2dda6-e623-4254-b987-2f6ff149f55c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.082544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2dda6-e623-4254-b987-2f6ff149f55c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.082544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2dda6-e623-4254-b987-2f6ff149f55c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.083970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 438/880] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:42.088191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2dda6-e623-4254-b987-2f6ff149f55c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:42.124900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 443/881] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:42 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:42.124900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3900d-0a11-4ce8-b658-0cedb27b0b59 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.124900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3900d-0a11-4ce8-b658-0cedb27b0b59 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.124900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3900d-0a11-4ce8-b658-0cedb27b0b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.143979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b7d3900d-0a11-4ce8-b658-0cedb27b0b59 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:42.145211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b7d3900d-0a11-4ce8-b658-0cedb27b0b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:42.150659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:42.203990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2dda6-e623-4254-b987-2f6ff149f55c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:42.230415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 444/882] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:42 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:42.247101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627313c-7cf1-42a6-a88c-03a2093de266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:42.247705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627313c-7cf1-42a6-a88c-03a2093de266 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.247976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627313c-7cf1-42a6-a88c-03a2093de266 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.248243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627313c-7cf1-42a6-a88c-03a2093de266 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.248899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627313c-7cf1-42a6-a88c-03a2093de266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:42.249198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627313c-7cf1-42a6-a88c-03a2093de266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:42.250679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627313c-7cf1-42a6-a88c-03a2093de266 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:42.264048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 439/883] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:42.270065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3900d-0a11-4ce8-b658-0cedb27b0b59 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:42.374433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:42.383345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3900d-0a11-4ce8-b658-0cedb27b0b59 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:42.410565 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627313c-7cf1-42a6-a88c-03a2093de266 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:42.418692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60231a-784b-447c-8526-16f2de4ac81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:42.419323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60231a-784b-447c-8526-16f2de4ac81d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.419596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60231a-784b-447c-8526-16f2de4ac81d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.419857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60231a-784b-447c-8526-16f2de4ac81d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.429013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60231a-784b-447c-8526-16f2de4ac81d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:42.430764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60231a-784b-447c-8526-16f2de4ac81d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-eb60231a-784b-447c-8526-16f2de4ac81d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:42.446754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:42.447694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:42.448604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:42.455895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 445/884] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:12:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:42.457005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60231a-784b-447c-8526-16f2de4ac81d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:42.534235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627313c-7cf1-42a6-a88c-03a2093de266 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:42.549675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:42.570092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60231a-784b-447c-8526-16f2de4ac81d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:42.571308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f341b01-9d9c-4414-aa66-9799f6475e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:42.571920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f341b01-9d9c-4414-aa66-9799f6475e8a None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.572194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f341b01-9d9c-4414-aa66-9799f6475e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.572466 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f341b01-9d9c-4414-aa66-9799f6475e8a None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.573156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f341b01-9d9c-4414-aa66-9799f6475e8a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:42.574510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f341b01-9d9c-4414-aa66-9799f6475e8a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:42.609022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b5332-89ca-4479-b183-20ac0b011a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:42.609022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b5332-89ca-4479-b183-20ac0b011a81 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.609022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b5332-89ca-4479-b183-20ac0b011a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.611629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b5332-89ca-4479-b183-20ac0b011a81 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.612212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b5332-89ca-4479-b183-20ac0b011a81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:42.612511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b5332-89ca-4479-b183-20ac0b011a81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:42.613991 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b5332-89ca-4479-b183-20ac0b011a81 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:42.633612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 446/885] 10.222.0.84 () {56 vars in 1149 bytes} [Wed Aug 24 11:12:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:42.634245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b5332-89ca-4479-b183-20ac0b011a81 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:42.651209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:42.676845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 440/886] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:12:42 2022] POST /identity/v3/services => generated 269 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:12:42.676845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f341b01-9d9c-4414-aa66-9799f6475e8a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:42.754333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b5332-89ca-4479-b183-20ac0b011a81 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:42.797396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575471a8-399d-4f32-8571-28ea0418b40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:42.798101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575471a8-399d-4f32-8571-28ea0418b40e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.798382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575471a8-399d-4f32-8571-28ea0418b40e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.798649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575471a8-399d-4f32-8571-28ea0418b40e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.799239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575471a8-399d-4f32-8571-28ea0418b40e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:42.800607 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575471a8-399d-4f32-8571-28ea0418b40e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:42.802871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f341b01-9d9c-4414-aa66-9799f6475e8a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-575471a8-399d-4f32-8571-28ea0418b40e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:42.810419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:42.811645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:42.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 447/887] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:42.813284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4161fee-2068-4d7e-ac79-518efc439413 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.813508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4161fee-2068-4d7e-ac79-518efc439413 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.813782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4161fee-2068-4d7e-ac79-518efc439413 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.815159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 448/888] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:42.817651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4161fee-2068-4d7e-ac79-518efc439413 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:42.848906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec287f7-be33-40b9-9aff-aec6c8d7e766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:42.851721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec287f7-be33-40b9-9aff-aec6c8d7e766 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.851721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec287f7-be33-40b9-9aff-aec6c8d7e766 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.851721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec287f7-be33-40b9-9aff-aec6c8d7e766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.896558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:42.903651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:42.923841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec287f7-be33-40b9-9aff-aec6c8d7e766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:42.925424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec287f7-be33-40b9-9aff-aec6c8d7e766 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:42.942280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4161fee-2068-4d7e-ac79-518efc439413 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:42.983710 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f64f26-4c54-49d4-bf1d-bbe554c3c56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:42.984316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f64f26-4c54-49d4-bf1d-bbe554c3c56c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.984587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f64f26-4c54-49d4-bf1d-bbe554c3c56c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.984883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f64f26-4c54-49d4-bf1d-bbe554c3c56c None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:42.985536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f64f26-4c54-49d4-bf1d-bbe554c3c56c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:42.985828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f64f26-4c54-49d4-bf1d-bbe554c3c56c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:42.987290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f64f26-4c54-49d4-bf1d-bbe554c3c56c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:42.992665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 449/889] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:42.993977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3375f6-890f-481c-a75a-ee7239b227d8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:42.994268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3375f6-890f-481c-a75a-ee7239b227d8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:42.994519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3375f6-890f-481c-a75a-ee7239b227d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:43.014163 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-1c3375f6-890f-481c-a75a-ee7239b227d8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:43.015334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1c3375f6-890f-481c-a75a-ee7239b227d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:43.051264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 441/890] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:42 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:12:43.051940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec287f7-be33-40b9-9aff-aec6c8d7e766 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:43.093945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 450/891] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:43.095152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3375f6-890f-481c-a75a-ee7239b227d8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:43.143028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:43.174015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec287f7-be33-40b9-9aff-aec6c8d7e766 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:43.175928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:43.215018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3375f6-890f-481c-a75a-ee7239b227d8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:43.227795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2bc21-b0b4-4aa1-85a4-eadae8e1efce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:43.228561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2bc21-b0b4-4aa1-85a4-eadae8e1efce None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:43.229141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2bc21-b0b4-4aa1-85a4-eadae8e1efce None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:43.229422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2bc21-b0b4-4aa1-85a4-eadae8e1efce None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:43.230074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2bc21-b0b4-4aa1-85a4-eadae8e1efce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:43.230380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2bc21-b0b4-4aa1-85a4-eadae8e1efce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:43.231754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2bc21-b0b4-4aa1-85a4-eadae8e1efce None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:43.245289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 442/892] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:43.261557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e54ba-6973-4b11-8095-128630d98f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:43.262603 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e54ba-6973-4b11-8095-128630d98f41 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:43.263104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e54ba-6973-4b11-8095-128630d98f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:43.263597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e54ba-6973-4b11-8095-128630d98f41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:43.276279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e54ba-6973-4b11-8095-128630d98f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:43.278190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e54ba-6973-4b11-8095-128630d98f41 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-131e54ba-6973-4b11-8095-128630d98f41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:43.295520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:43.296531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:43.297446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:43.297446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:43.297446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:43.299744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 451/893] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:43.306752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2bc21-b0b4-4aa1-85a4-eadae8e1efce None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:43.378821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e54ba-6973-4b11-8095-128630d98f41 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:43.436062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2bc21-b0b4-4aa1-85a4-eadae8e1efce None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:43.472986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a089615-7491-4179-bebe-73b547966c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:43.473586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a089615-7491-4179-bebe-73b547966c00 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:43.474851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a089615-7491-4179-bebe-73b547966c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:43.474851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a089615-7491-4179-bebe-73b547966c00 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:43.474851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a089615-7491-4179-bebe-73b547966c00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:43.475069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a089615-7491-4179-bebe-73b547966c00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:43.476495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a089615-7491-4179-bebe-73b547966c00 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:43.486597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:43.492641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 443/894] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:43.498108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a089615-7491-4179-bebe-73b547966c00 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:43.514878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e54ba-6973-4b11-8095-128630d98f41 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:43.535538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:43.553613 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f74239-145e-46ae-9e9d-343be6ecc95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:43.554241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f74239-145e-46ae-9e9d-343be6ecc95c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:43.554519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f74239-145e-46ae-9e9d-343be6ecc95c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:43.554791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f74239-145e-46ae-9e9d-343be6ecc95c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:43.555373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f74239-145e-46ae-9e9d-343be6ecc95c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:43.556962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f74239-145e-46ae-9e9d-343be6ecc95c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-d5f74239-145e-46ae-9e9d-343be6ecc95c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:43.566814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:43.567813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:43.568632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 452/895] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:43.574305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f74239-145e-46ae-9e9d-343be6ecc95c None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:43.637330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a089615-7491-4179-bebe-73b547966c00 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:43.690809 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c7823-b23f-40fc-81c3-d85482daa83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:43.691806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c7823-b23f-40fc-81c3-d85482daa83c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:43.692288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c7823-b23f-40fc-81c3-d85482daa83c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:43.692761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c7823-b23f-40fc-81c3-d85482daa83c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:43.694665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f74239-145e-46ae-9e9d-343be6ecc95c None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:43.700673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c7823-b23f-40fc-81c3-d85482daa83c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:43.702425 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c7823-b23f-40fc-81c3-d85482daa83c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e66c7823-b23f-40fc-81c3-d85482daa83c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:43.714754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:43.715737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:43.716605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:43.716605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:43.716605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:43.716605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:43.716605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:43.716605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:43.716605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:43.716605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:43.719820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 444/896] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:43.725356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c7823-b23f-40fc-81c3-d85482daa83c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:43.736889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359dfa1-ac3a-4d9e-bef4-4c617ed8e5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:43.738303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359dfa1-ac3a-4d9e-bef4-4c617ed8e5aa None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:43.738793 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359dfa1-ac3a-4d9e-bef4-4c617ed8e5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:43.739271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359dfa1-ac3a-4d9e-bef4-4c617ed8e5aa None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:43.740483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359dfa1-ac3a-4d9e-bef4-4c617ed8e5aa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:43.741022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359dfa1-ac3a-4d9e-bef4-4c617ed8e5aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:43.743299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359dfa1-ac3a-4d9e-bef4-4c617ed8e5aa None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:43.750251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 453/897] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:43.761806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359dfa1-ac3a-4d9e-bef4-4c617ed8e5aa None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:43.802413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:43.829525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:43.857191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c7823-b23f-40fc-81c3-d85482daa83c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:43.901317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25886ff9-b84e-4846-9259-11bd07e0fb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:43.902232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25886ff9-b84e-4846-9259-11bd07e0fb45 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:43.902662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25886ff9-b84e-4846-9259-11bd07e0fb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:43.903069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25886ff9-b84e-4846-9259-11bd07e0fb45 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:43.903902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25886ff9-b84e-4846-9259-11bd07e0fb45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:43.904386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25886ff9-b84e-4846-9259-11bd07e0fb45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:43.904949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359dfa1-ac3a-4d9e-bef4-4c617ed8e5aa None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:43.906162 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25886ff9-b84e-4846-9259-11bd07e0fb45 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:43.925179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 445/898] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:43.926759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25886ff9-b84e-4846-9259-11bd07e0fb45 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:43.940653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3c8f8-bedb-44d2-9596-0fc5c85a74cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:43.941412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3c8f8-bedb-44d2-9596-0fc5c85a74cf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:43.941916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3c8f8-bedb-44d2-9596-0fc5c85a74cf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:43.942034 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3c8f8-bedb-44d2-9596-0fc5c85a74cf None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:43.942694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3c8f8-bedb-44d2-9596-0fc5c85a74cf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:43.943133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3c8f8-bedb-44d2-9596-0fc5c85a74cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:43.944531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3c8f8-bedb-44d2-9596-0fc5c85a74cf None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:43.954387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 454/899] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:43.955650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3c8f8-bedb-44d2-9596-0fc5c85a74cf None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:44.057400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25886ff9-b84e-4846-9259-11bd07e0fb45 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:44.079534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:44.090289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3c8f8-bedb-44d2-9596-0fc5c85a74cf None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:44.100300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47c943-c47f-4a0a-8b63-f7616ab7fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:44.100940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c943-c47f-4a0a-8b63-f7616ab7fa71 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:44.101774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c943-c47f-4a0a-8b63-f7616ab7fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:44.102142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47c943-c47f-4a0a-8b63-f7616ab7fa71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:44.103049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47c943-c47f-4a0a-8b63-f7616ab7fa71 None admin] RBAC: Adding query filter params (role.id=b2da7bb417d94f45a2a4e8a2befa0988, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=164f3bcc6363445da191bd1b78499a53) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:44.103350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47c943-c47f-4a0a-8b63-f7616ab7fa71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:44.105286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47c943-c47f-4a0a-8b63-f7616ab7fa71 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:44.105526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:44.117381 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 446/900] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/role_assignments?user.id=164f3bcc6363445da191bd1b78499a53&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=b2da7bb417d94f45a2a4e8a2befa0988 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:44.141166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbaacc-82c9-4721-a597-71a8a781f42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:44.142055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbaacc-82c9-4721-a597-71a8a781f42c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:44.142485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbaacc-82c9-4721-a597-71a8a781f42c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:44.142901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbaacc-82c9-4721-a597-71a8a781f42c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:44.143764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbaacc-82c9-4721-a597-71a8a781f42c None admin] RBAC: Adding query filter params (interface=public, service_id=29be825dc7c74043a181cb26f837aa3e, region_id=RegionOne) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:44.144249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbaacc-82c9-4721-a597-71a8a781f42c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:44.145928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbaacc-82c9-4721-a597-71a8a781f42c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:44.152945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 455/901] 10.222.0.84 () {56 vars in 1251 bytes} [Wed Aug 24 11:12:43 2022] GET /identity/v3/endpoints?service_id=29be825dc7c74043a181cb26f837aa3e&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:44.159208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47c943-c47f-4a0a-8b63-f7616ab7fa71 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:44.277964 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47c943-c47f-4a0a-8b63-f7616ab7fa71 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:44.318200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b56e9c-a1fd-45d7-945e-7da76e8e071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:44.318835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b56e9c-a1fd-45d7-945e-7da76e8e071e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:44.319117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b56e9c-a1fd-45d7-945e-7da76e8e071e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:44.319389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b56e9c-a1fd-45d7-945e-7da76e8e071e None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:44.320215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b56e9c-a1fd-45d7-945e-7da76e8e071e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:44.321844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b56e9c-a1fd-45d7-945e-7da76e8e071e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:44.328228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 447/902] 10.222.0.84 () {56 vars in 1087 bytes} [Wed Aug 24 11:12:44 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:12:44.735972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874d57b-d557-48f3-a23d-4ca0078d2b90 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:44.736304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874d57b-d557-48f3-a23d-4ca0078d2b90 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:44.736541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874d57b-d557-48f3-a23d-4ca0078d2b90 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:44.738190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 456/903] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:44.744611 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd0dd2-ebcb-40cd-9618-cd5bd88175db None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:44.744924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd0dd2-ebcb-40cd-9618-cd5bd88175db None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:44.745201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd0dd2-ebcb-40cd-9618-cd5bd88175db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:44.766527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-abdd0dd2-ebcb-40cd-9618-cd5bd88175db None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:44.768094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-abdd0dd2-ebcb-40cd-9618-cd5bd88175db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:44.816868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:44.844160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 448/904] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:45.119691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874d57b-d557-48f3-a23d-4ca0078d2b90 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:45.226900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874d57b-d557-48f3-a23d-4ca0078d2b90 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:45.261836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9981ca5-b676-4f09-bcaa-80d4868cb22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:45.262936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981ca5-b676-4f09-bcaa-80d4868cb22b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:45.263357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981ca5-b676-4f09-bcaa-80d4868cb22b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:45.263815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9981ca5-b676-4f09-bcaa-80d4868cb22b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:45.264557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9981ca5-b676-4f09-bcaa-80d4868cb22b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:45.266139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9981ca5-b676-4f09-bcaa-80d4868cb22b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-e9981ca5-b676-4f09-bcaa-80d4868cb22b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:45.275894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:45.277076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:12:45.278149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:45.285040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 457/905] 10.222.0.84 () {56 vars in 1132 bytes} [Wed Aug 24 11:12:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:45.286819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd0dd2-ebcb-40cd-9618-cd5bd88175db None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:45.318731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979704-074b-4c43-9a4a-7a21183efa57 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:45.319251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979704-074b-4c43-9a4a-7a21183efa57 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:45.320388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979704-074b-4c43-9a4a-7a21183efa57 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:45.322878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 458/906] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:12:45.327978 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ef69d-e360-4baf-8c66-72b0c84d4e2f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:45.328409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ef69d-e360-4baf-8c66-72b0c84d4e2f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:45.328858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ef69d-e360-4baf-8c66-72b0c84d4e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:45.352453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-808ef69d-e360-4baf-8c66-72b0c84d4e2f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:45.354498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-808ef69d-e360-4baf-8c66-72b0c84d4e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:45.393875 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:45.407566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd0dd2-ebcb-40cd-9618-cd5bd88175db None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:45.445270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 459/907] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:45.449037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768084f-165b-4fc4-9ca7-83aeb3a04fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:45.449639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768084f-165b-4fc4-9ca7-83aeb3a04fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:45.449910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768084f-165b-4fc4-9ca7-83aeb3a04fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:45.450176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768084f-165b-4fc4-9ca7-83aeb3a04fc3 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:45.450798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768084f-165b-4fc4-9ca7-83aeb3a04fc3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:45.451090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768084f-165b-4fc4-9ca7-83aeb3a04fc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:45.452443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768084f-165b-4fc4-9ca7-83aeb3a04fc3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:45.462542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 449/908] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:45.469206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ef69d-e360-4baf-8c66-72b0c84d4e2f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:45.579911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ef69d-e360-4baf-8c66-72b0c84d4e2f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:45.615207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ddb64-66d1-40b2-9d95-887f5827abf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:45.615808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ddb64-66d1-40b2-9d95-887f5827abf1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:45.616080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ddb64-66d1-40b2-9d95-887f5827abf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:45.616346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ddb64-66d1-40b2-9d95-887f5827abf1 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:45.617097 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ddb64-66d1-40b2-9d95-887f5827abf1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:45.617393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ddb64-66d1-40b2-9d95-887f5827abf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:45.618750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ddb64-66d1-40b2-9d95-887f5827abf1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:45.628542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 460/909] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:45.630071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768084f-165b-4fc4-9ca7-83aeb3a04fc3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:45.675704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:45.737945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768084f-165b-4fc4-9ca7-83aeb3a04fc3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:45.738768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ddb64-66d1-40b2-9d95-887f5827abf1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:45.782284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:45.783590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f977a0-fcd4-4369-b689-ad49077969fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:45.784551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f977a0-fcd4-4369-b689-ad49077969fe None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:45.784852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f977a0-fcd4-4369-b689-ad49077969fe None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:45.785128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f977a0-fcd4-4369-b689-ad49077969fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:45.786108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f977a0-fcd4-4369-b689-ad49077969fe None admin] RBAC: Adding query filter params (interface=public, service_id=88e21cad71c846e6bba6f66f6adef464, region_id=RegionOne) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:45.786414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f977a0-fcd4-4369-b689-ad49077969fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:45.788035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f977a0-fcd4-4369-b689-ad49077969fe None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:45.796884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 450/910] 10.222.0.84 () {56 vars in 1251 bytes} [Wed Aug 24 11:12:45 2022] GET /identity/v3/endpoints?service_id=88e21cad71c846e6bba6f66f6adef464&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:45.801591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f977a0-fcd4-4369-b689-ad49077969fe None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:45.850334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ddb64-66d1-40b2-9d95-887f5827abf1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:45.887154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c884131-6588-4e06-8a62-8517fd578a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:45.887761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c884131-6588-4e06-8a62-8517fd578a25 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:45.888037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c884131-6588-4e06-8a62-8517fd578a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:45.888306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c884131-6588-4e06-8a62-8517fd578a25 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:45.889010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c884131-6588-4e06-8a62-8517fd578a25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:45.890384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c884131-6588-4e06-8a62-8517fd578a25 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:45.928644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f977a0-fcd4-4369-b689-ad49077969fe None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:45.932264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:45.958231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 461/911] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:12:45 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:12:45.992980 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cb902-cff2-43e0-a3f8-0342f1fa1bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:45.994006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cb902-cff2-43e0-a3f8-0342f1fa1bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:45.994483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cb902-cff2-43e0-a3f8-0342f1fa1bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:45.995002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cb902-cff2-43e0-a3f8-0342f1fa1bb0 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:45.995982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cb902-cff2-43e0-a3f8-0342f1fa1bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:45.997855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cb902-cff2-43e0-a3f8-0342f1fa1bb0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:46.012640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 451/912] 10.222.0.84 () {56 vars in 1087 bytes} [Wed Aug 24 11:12:45 2022] GET /identity/v3/services => generated 1497 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:12:46.192027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36d0b0-d9e9-400a-bd19-08ef4060c5d9 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:46.192506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36d0b0-d9e9-400a-bd19-08ef4060c5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:46.192506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36d0b0-d9e9-400a-bd19-08ef4060c5d9 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:46.194122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 462/913] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:46.200019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34425399-469f-4294-b2b5-46ee7cffc9fe None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:46.200306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34425399-469f-4294-b2b5-46ee7cffc9fe None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:46.200582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34425399-469f-4294-b2b5-46ee7cffc9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:46.220582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-34425399-469f-4294-b2b5-46ee7cffc9fe None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:46.221803 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-34425399-469f-4294-b2b5-46ee7cffc9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:46.298085 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 452/914] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:46.365326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b3373-264d-4705-b42e-d87b3990fe12 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:46.365640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b3373-264d-4705-b42e-d87b3990fe12 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:46.365872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b3373-264d-4705-b42e-d87b3990fe12 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:46.368375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 463/915] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:46.373375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae52ff9-c9ab-479d-8025-05ed9be128d0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:46.373654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae52ff9-c9ab-479d-8025-05ed9be128d0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:46.373923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae52ff9-c9ab-479d-8025-05ed9be128d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:46.392650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8ae52ff9-c9ab-479d-8025-05ed9be128d0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:46.393869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8ae52ff9-c9ab-479d-8025-05ed9be128d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:46.463376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 453/916] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:46.588153 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b3373-264d-4705-b42e-d87b3990fe12 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:46.685505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:46.699920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b3373-264d-4705-b42e-d87b3990fe12 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:46.736095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae52ff9-c9ab-479d-8025-05ed9be128d0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:46.737106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5fd88-f140-42e2-8448-aeae78a1a1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:46.738957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5fd88-f140-42e2-8448-aeae78a1a1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:46.739664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5fd88-f140-42e2-8448-aeae78a1a1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:46.741387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5fd88-f140-42e2-8448-aeae78a1a1d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:46.756105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5fd88-f140-42e2-8448-aeae78a1a1d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-e0f5fd88-f140-42e2-8448-aeae78a1a1d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:46.762004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:46.762910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:46.763879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 464/917] 10.222.0.84 () {56 vars in 1120 bytes} [Wed Aug 24 11:12:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:46.772896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5fd88-f140-42e2-8448-aeae78a1a1d7 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:46.779861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:46.863325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae52ff9-c9ab-479d-8025-05ed9be128d0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:46.894357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5fd88-f140-42e2-8448-aeae78a1a1d7 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:46.904259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776ede9-26a7-4bfb-8017-fbc95d5a0a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:46.905253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ede9-26a7-4bfb-8017-fbc95d5a0a9c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:46.905794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ede9-26a7-4bfb-8017-fbc95d5a0a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:46.906243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ede9-26a7-4bfb-8017-fbc95d5a0a9c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:46.907092 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776ede9-26a7-4bfb-8017-fbc95d5a0a9c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:46.908747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776ede9-26a7-4bfb-8017-fbc95d5a0a9c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-8776ede9-26a7-4bfb-8017-fbc95d5a0a9c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:46.918382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:46.919343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 11:12:46.920168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:46.927258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 454/918] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:12:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:46.927844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d374d2e-2e7b-4ee4-9dd6-90e1a3be9a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:46.928438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d374d2e-2e7b-4ee4-9dd6-90e1a3be9a09 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:46.928604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776ede9-26a7-4bfb-8017-fbc95d5a0a9c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:46.928707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d374d2e-2e7b-4ee4-9dd6-90e1a3be9a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:46.928987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d374d2e-2e7b-4ee4-9dd6-90e1a3be9a09 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:46.930340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d374d2e-2e7b-4ee4-9dd6-90e1a3be9a09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:46.930630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d374d2e-2e7b-4ee4-9dd6-90e1a3be9a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:46.932220 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d374d2e-2e7b-4ee4-9dd6-90e1a3be9a09 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:46.940774 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 465/919] 10.222.0.84 () {56 vars in 1115 bytes} [Wed Aug 24 11:12:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:46.946742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d374d2e-2e7b-4ee4-9dd6-90e1a3be9a09 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:46.963245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:47.032635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:47.046358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776ede9-26a7-4bfb-8017-fbc95d5a0a9c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:47.051515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d374d2e-2e7b-4ee4-9dd6-90e1a3be9a09 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:47.084605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821657-b20b-4950-b7c6-47c8482226b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:47.085563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821657-b20b-4950-b7c6-47c8482226b9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.086632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821657-b20b-4950-b7c6-47c8482226b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.087055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821657-b20b-4950-b7c6-47c8482226b9 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.087887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05821657-b20b-4950-b7c6-47c8482226b9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:47.088340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05821657-b20b-4950-b7c6-47c8482226b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:47.089144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c905ee-6cb3-4a29-8b23-ca2fa39df5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:47.089876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c905ee-6cb3-4a29-8b23-ca2fa39df5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.090219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c905ee-6cb3-4a29-8b23-ca2fa39df5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.090555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c905ee-6cb3-4a29-8b23-ca2fa39df5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.090671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05821657-b20b-4950-b7c6-47c8482226b9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:47.096832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 455/920] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:12:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:47.105327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c905ee-6cb3-4a29-8b23-ca2fa39df5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:47.105327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c905ee-6cb3-4a29-8b23-ca2fa39df5e8 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:47.106828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821657-b20b-4950-b7c6-47c8482226b9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-43c905ee-6cb3-4a29-8b23-ca2fa39df5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:47.120916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:47.121888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:47.122798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:47.122798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:47.122798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:47.122798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 466/921] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:47.127775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c905ee-6cb3-4a29-8b23-ca2fa39df5e8 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:47.198474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:47.221596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821657-b20b-4950-b7c6-47c8482226b9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:47.237122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c905ee-6cb3-4a29-8b23-ca2fa39df5e8 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:47.261372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e862e44-5eb4-457e-a988-7a76b800ce7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:47.262321 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e862e44-5eb4-457e-a988-7a76b800ce7b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.262878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e862e44-5eb4-457e-a988-7a76b800ce7b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.263328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e862e44-5eb4-457e-a988-7a76b800ce7b None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.264214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e862e44-5eb4-457e-a988-7a76b800ce7b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:47.264698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e862e44-5eb4-457e-a988-7a76b800ce7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:47.266563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e862e44-5eb4-457e-a988-7a76b800ce7b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:47.277486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f7f80-c152-43d0-8992-6a752875ea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:47.277486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f7f80-c152-43d0-8992-6a752875ea51 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.277486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f7f80-c152-43d0-8992-6a752875ea51 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.277486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f7f80-c152-43d0-8992-6a752875ea51 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.277843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f7f80-c152-43d0-8992-6a752875ea51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:47.277843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f7f80-c152-43d0-8992-6a752875ea51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:47.278132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 456/922] 10.222.0.84 () {56 vars in 1114 bytes} [Wed Aug 24 11:12:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:47.279130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e862e44-5eb4-457e-a988-7a76b800ce7b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:47.279562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f7f80-c152-43d0-8992-6a752875ea51 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:47.293736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 467/923] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:47.299167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f7f80-c152-43d0-8992-6a752875ea51 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:47.301527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:47.391687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e862e44-5eb4-457e-a988-7a76b800ce7b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:47.405754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f7f80-c152-43d0-8992-6a752875ea51 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:47.429560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068807fd-c488-41ad-b22a-98d1a565ac74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:47.430483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068807fd-c488-41ad-b22a-98d1a565ac74 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.431034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068807fd-c488-41ad-b22a-98d1a565ac74 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.431587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068807fd-c488-41ad-b22a-98d1a565ac74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.432598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068807fd-c488-41ad-b22a-98d1a565ac74 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:47.434197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:47.434742 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068807fd-c488-41ad-b22a-98d1a565ac74 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:47.442682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962accfa-45f7-4f10-8208-6d41caf42658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:47.443390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962accfa-45f7-4f10-8208-6d41caf42658 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.443659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962accfa-45f7-4f10-8208-6d41caf42658 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.443923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962accfa-45f7-4f10-8208-6d41caf42658 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.451745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962accfa-45f7-4f10-8208-6d41caf42658 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:47.453400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962accfa-45f7-4f10-8208-6d41caf42658 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-962accfa-45f7-4f10-8208-6d41caf42658 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:47.465352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:47.466355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:47.467306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:47.468865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 468/924] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:12:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:47.472328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962accfa-45f7-4f10-8208-6d41caf42658 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:47.539582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 457/925] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:12:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:47.590924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962accfa-45f7-4f10-8208-6d41caf42658 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:47.636889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8aa02-22c3-4702-929c-ecea19a36e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:47.637514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8aa02-22c3-4702-929c-ecea19a36e57 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.637786 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8aa02-22c3-4702-929c-ecea19a36e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.638048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8aa02-22c3-4702-929c-ecea19a36e57 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.638775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8aa02-22c3-4702-929c-ecea19a36e57 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:47.639072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8aa02-22c3-4702-929c-ecea19a36e57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:47.640520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8aa02-22c3-4702-929c-ecea19a36e57 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:47.662525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 469/926] 10.222.0.84 () {56 vars in 1149 bytes} [Wed Aug 24 11:12:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:47.669978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068807fd-c488-41ad-b22a-98d1a565ac74 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:47.750517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:47.786196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068807fd-c488-41ad-b22a-98d1a565ac74 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:47.823015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f8224-1dba-4bd8-a043-be9b26e3fca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:47.823645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f8224-1dba-4bd8-a043-be9b26e3fca7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.823920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f8224-1dba-4bd8-a043-be9b26e3fca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.824185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f8224-1dba-4bd8-a043-be9b26e3fca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.835243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f8224-1dba-4bd8-a043-be9b26e3fca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:47.836956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f8224-1dba-4bd8-a043-be9b26e3fca7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-492f8224-1dba-4bd8-a043-be9b26e3fca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:47.852746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:47.853706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:47.854596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:47.854596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:47.854596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:47.854596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 458/927] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:47.860358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8aa02-22c3-4702-929c-ecea19a36e57 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:47.904144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:47.958443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd52cd-3bb6-4d13-a99c-2f9a57217867 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.958778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd52cd-3bb6-4d13-a99c-2f9a57217867 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.958990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd52cd-3bb6-4d13-a99c-2f9a57217867 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.960477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 459/928] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:47.966562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125a899-a68c-45e9-b3fe-eb58f0260cce None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:47.967186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125a899-a68c-45e9-b3fe-eb58f0260cce None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:47.967677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125a899-a68c-45e9-b3fe-eb58f0260cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:47.969791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8aa02-22c3-4702-929c-ecea19a36e57 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:47.989531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1125a899-a68c-45e9-b3fe-eb58f0260cce None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:47.991047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1125a899-a68c-45e9-b3fe-eb58f0260cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:48.007454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05481230-f4c8-4605-a60e-57434e68e21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:48.008059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05481230-f4c8-4605-a60e-57434e68e21e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.008339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05481230-f4c8-4605-a60e-57434e68e21e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.008616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05481230-f4c8-4605-a60e-57434e68e21e None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.009659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05481230-f4c8-4605-a60e-57434e68e21e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:48.010090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05481230-f4c8-4605-a60e-57434e68e21e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:48.012181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05481230-f4c8-4605-a60e-57434e68e21e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:48.028701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 470/929] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:48.033962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05481230-f4c8-4605-a60e-57434e68e21e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:48.073087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 460/930] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:48.107751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2f05-4b1a-4b0e-b969-9694235debd4 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.108719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2f05-4b1a-4b0e-b969-9694235debd4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.109222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2f05-4b1a-4b0e-b969-9694235debd4 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.111030 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 461/931] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:12:48.118802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c26eae-f31d-42b8-b277-ae650f54d522 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.119076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c26eae-f31d-42b8-b277-ae650f54d522 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.119357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c26eae-f31d-42b8-b277-ae650f54d522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.142277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a0c26eae-f31d-42b8-b277-ae650f54d522 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:48.143792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a0c26eae-f31d-42b8-b277-ae650f54d522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:48.147122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05481230-f4c8-4605-a60e-57434e68e21e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:48.169211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:48.189496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b6a1e-85a2-4b59-9255-001d6709ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:48.190931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b6a1e-85a2-4b59-9255-001d6709ca2b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.191206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b6a1e-85a2-4b59-9255-001d6709ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.191613 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b6a1e-85a2-4b59-9255-001d6709ca2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.199092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b6a1e-85a2-4b59-9255-001d6709ca2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:48.201418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b6a1e-85a2-4b59-9255-001d6709ca2b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:48.207701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-050b6a1e-85a2-4b59-9255-001d6709ca2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:48.216257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:48.221089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:48.221954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:48.221954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:48.221954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:48.221954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:48.221954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:48.221954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:48.221954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:48.221954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:48.221954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 471/932] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:48.223657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b6a1e-85a2-4b59-9255-001d6709ca2b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:48.234443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 462/933] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:48.334105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b6a1e-85a2-4b59-9255-001d6709ca2b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:48.353953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c26eae-f31d-42b8-b277-ae650f54d522 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:48.374956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b54be-7919-459c-9989-9ead6a9fbfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:48.376183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b54be-7919-459c-9989-9ead6a9fbfcc None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.376609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b54be-7919-459c-9989-9ead6a9fbfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.377339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b54be-7919-459c-9989-9ead6a9fbfcc None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.378210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b54be-7919-459c-9989-9ead6a9fbfcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:48.378777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b54be-7919-459c-9989-9ead6a9fbfcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:48.380968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b54be-7919-459c-9989-9ead6a9fbfcc None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:48.397433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 472/934] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:48.404314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b54be-7919-459c-9989-9ead6a9fbfcc None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:48.431606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:48.475162 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c26eae-f31d-42b8-b277-ae650f54d522 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:48.522048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f646b-0737-4c84-a760-1ae8538e5d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:48.523515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f646b-0737-4c84-a760-1ae8538e5d00 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.523994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f646b-0737-4c84-a760-1ae8538e5d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.524441 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f646b-0737-4c84-a760-1ae8538e5d00 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.524877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b54be-7919-459c-9989-9ead6a9fbfcc None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:48.525401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f646b-0737-4c84-a760-1ae8538e5d00 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:48.527154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f646b-0737-4c84-a760-1ae8538e5d00 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-4c3f646b-0737-4c84-a760-1ae8538e5d00 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:48.536145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:48.537350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:48.538216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 463/935] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:48.539600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f646b-0737-4c84-a760-1ae8538e5d00 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:48.565036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4454f-5671-437e-b1c4-a8c1878c1d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:48.565701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4454f-5671-437e-b1c4-a8c1878c1d85 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.565979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4454f-5671-437e-b1c4-a8c1878c1d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.566250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4454f-5671-437e-b1c4-a8c1878c1d85 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/164f3bcc6363445da191bd1b78499a53/roles/b2da7bb417d94f45a2a4e8a2befa0988` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.568548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4454f-5671-437e-b1c4-a8c1878c1d85 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=164f3bcc6363445da191bd1b78499a53, role_id=b2da7bb417d94f45a2a4e8a2befa0988)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:48.570531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d4454f-5671-437e-b1c4-a8c1878c1d85 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:48.593717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:48.643463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f646b-0737-4c84-a760-1ae8538e5d00 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:48.656364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 473/936] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:12:48 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/164f3bcc6363445da191bd1b78499a53/roles/b2da7bb417d94f45a2a4e8a2befa0988 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:12:48.657556 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4454f-5671-437e-b1c4-a8c1878c1d85 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:48.707342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3297ef8-d84e-48f8-8d60-1ada1d001c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:48.708112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3297ef8-d84e-48f8-8d60-1ada1d001c29 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.708395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3297ef8-d84e-48f8-8d60-1ada1d001c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.708666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3297ef8-d84e-48f8-8d60-1ada1d001c29 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.709302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3297ef8-d84e-48f8-8d60-1ada1d001c29 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:48.711402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3297ef8-d84e-48f8-8d60-1ada1d001c29 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a3297ef8-d84e-48f8-8d60-1ada1d001c29 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:48.722159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:48.723352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 11:12:48.724524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:48.728831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 464/937] 10.222.0.84 () {56 vars in 1132 bytes} [Wed Aug 24 11:12:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:48.730378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3297ef8-d84e-48f8-8d60-1ada1d001c29 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:48.764764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:48.787159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4454f-5671-437e-b1c4-a8c1878c1d85 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:48.829058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557434b8-34a5-44ed-a05c-246d674badc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:48.830029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557434b8-34a5-44ed-a05c-246d674badc1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.830249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557434b8-34a5-44ed-a05c-246d674badc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.830612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557434b8-34a5-44ed-a05c-246d674badc1 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.831338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557434b8-34a5-44ed-a05c-246d674badc1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:48.831732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557434b8-34a5-44ed-a05c-246d674badc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:48.833240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557434b8-34a5-44ed-a05c-246d674badc1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:48.839455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 474/938] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:48.844839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557434b8-34a5-44ed-a05c-246d674badc1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:48.860849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3297ef8-d84e-48f8-8d60-1ada1d001c29 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:48.880175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:48.904639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4b73-ef97-4dcb-b1ed-74adbaa68bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:48.905570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4b73-ef97-4dcb-b1ed-74adbaa68bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.906020 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4b73-ef97-4dcb-b1ed-74adbaa68bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.906430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4b73-ef97-4dcb-b1ed-74adbaa68bc9 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.907200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4b73-ef97-4dcb-b1ed-74adbaa68bc9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:48.907746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4b73-ef97-4dcb-b1ed-74adbaa68bc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:48.909401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4b73-ef97-4dcb-b1ed-74adbaa68bc9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:48.915385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 465/939] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:48.922926 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4b73-ef97-4dcb-b1ed-74adbaa68bc9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:48.953980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557434b8-34a5-44ed-a05c-246d674badc1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:48.989646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7aba92-169e-408b-a2d3-30008f3ab864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:48.990439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7aba92-169e-408b-a2d3-30008f3ab864 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:48.990807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7aba92-169e-408b-a2d3-30008f3ab864 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:48.991186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7aba92-169e-408b-a2d3-30008f3ab864 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:48.992428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7aba92-169e-408b-a2d3-30008f3ab864 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:48.992866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7aba92-169e-408b-a2d3-30008f3ab864 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:48.994350 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7aba92-169e-408b-a2d3-30008f3ab864 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:48.999844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 475/940] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:49.005341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7aba92-169e-408b-a2d3-30008f3ab864 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:49.019483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:49.037635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4b73-ef97-4dcb-b1ed-74adbaa68bc9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:49.076514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf20cc-ca89-41af-942c-346f65f727d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:49.077525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf20cc-ca89-41af-942c-346f65f727d1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:49.078048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf20cc-ca89-41af-942c-346f65f727d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:49.078467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf20cc-ca89-41af-942c-346f65f727d1 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:49.079258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf20cc-ca89-41af-942c-346f65f727d1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:49.079709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf20cc-ca89-41af-942c-346f65f727d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:49.081368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf20cc-ca89-41af-942c-346f65f727d1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:49.092397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 466/941] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:49.093615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf20cc-ca89-41af-942c-346f65f727d1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:49.114431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7aba92-169e-408b-a2d3-30008f3ab864 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:49.141522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:49.150564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eb559-73ac-4ff1-b37d-6b908196ba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:49.151229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eb559-73ac-4ff1-b37d-6b908196ba6b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:49.151653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eb559-73ac-4ff1-b37d-6b908196ba6b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:49.151900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eb559-73ac-4ff1-b37d-6b908196ba6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:49.152596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eb559-73ac-4ff1-b37d-6b908196ba6b None admin] RBAC: Adding query filter params (interface=public, service_id=f7f4e3aa6e2f4bc483b019cecb2f82a1, region_id=RegionOne) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:49.152923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eb559-73ac-4ff1-b37d-6b908196ba6b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:49.163169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eb559-73ac-4ff1-b37d-6b908196ba6b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:49.163169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 476/942] 10.222.0.84 () {56 vars in 1251 bytes} [Wed Aug 24 11:12:49 2022] GET /identity/v3/endpoints?service_id=f7f4e3aa6e2f4bc483b019cecb2f82a1&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:49.166743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eb559-73ac-4ff1-b37d-6b908196ba6b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:49.232065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf20cc-ca89-41af-942c-346f65f727d1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:49.243604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:49.274989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eb559-73ac-4ff1-b37d-6b908196ba6b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:49.289049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b45825-fdcc-4d7a-b3c6-68971f273e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:49.289049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b45825-fdcc-4d7a-b3c6-68971f273e21 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:49.289049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b45825-fdcc-4d7a-b3c6-68971f273e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:49.289049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b45825-fdcc-4d7a-b3c6-68971f273e21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:49.289465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b45825-fdcc-4d7a-b3c6-68971f273e21 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:49.289465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b45825-fdcc-4d7a-b3c6-68971f273e21 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:49.310036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76bf62-979c-434c-8744-bda0bb7dbe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:49.310790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76bf62-979c-434c-8744-bda0bb7dbe55 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:49.311134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76bf62-979c-434c-8744-bda0bb7dbe55 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:49.311473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76bf62-979c-434c-8744-bda0bb7dbe55 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:49.312142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76bf62-979c-434c-8744-bda0bb7dbe55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:49.313593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76bf62-979c-434c-8744-bda0bb7dbe55 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:49.322172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 477/943] 10.222.0.84 () {56 vars in 1087 bytes} [Wed Aug 24 11:12:49 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:12:49.422819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 467/944] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:12:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:49.700816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec2b51-6122-4803-b71e-8301bb327212 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:49.701016 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec2b51-6122-4803-b71e-8301bb327212 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:49.701285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec2b51-6122-4803-b71e-8301bb327212 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:49.702915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 478/945] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:49.708380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50b9ca-fd36-4cb0-b083-002f43b1d229 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:49.708670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50b9ca-fd36-4cb0-b083-002f43b1d229 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:49.708976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50b9ca-fd36-4cb0-b083-002f43b1d229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:49.727161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2a50b9ca-fd36-4cb0-b083-002f43b1d229 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:49.728455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2a50b9ca-fd36-4cb0-b083-002f43b1d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:49.811911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 468/946] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:50.080240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec2b51-6122-4803-b71e-8301bb327212 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:50.185451 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec2b51-6122-4803-b71e-8301bb327212 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:50.221209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:50.221809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:50.222080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:50.222344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:50.222989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:50.224406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:50.227632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WARNING keystone.server.flask.common [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 11:12:50.231868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:50.311859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.sql.core [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 11:12:50.311859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 11:12:50.311859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [parameters: {'id': '1af93c46ff774ea99e0bf32edcf8cc0a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 11:12:50.311859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81707) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1af93c46ff774ea99e0bf32edcf8cc0a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1af93c46ff774ea99e0bf32edcf8cc0a. Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:12:50.329832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application result.read() Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 11:12:50.330781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application yield resource Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise self.value Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application session.commit() Aug 24 11:12:50.331831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self.session.flush() Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application compat.raise_( Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 11:12:50.332696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 11:12:50.333648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application result.read() Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:12:50.334438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [parameters: {'id': '1af93c46ff774ea99e0bf32edcf8cc0a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:50.335299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:50.336226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1af93c46ff774ea99e0bf32edcf8cc0a. Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:50.337343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 479/947] 10.222.0.84 () {60 vars in 1140 bytes} [Wed Aug 24 11:12:50 2022] POST /identity/v3/projects => generated 262 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:50.338441 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50b9ca-fd36-4cb0-b083-002f43b1d229 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:50.364167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:50.446288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50b9ca-fd36-4cb0-b083-002f43b1d229 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:50.485137 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f852b5-8c30-462c-aa4d-362111b4a9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:50.485765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f852b5-8c30-462c-aa4d-362111b4a9db None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:50.486037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f852b5-8c30-462c-aa4d-362111b4a9db None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:50.486304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f852b5-8c30-462c-aa4d-362111b4a9db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:50.493450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f852b5-8c30-462c-aa4d-362111b4a9db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:50.494904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f852b5-8c30-462c-aa4d-362111b4a9db None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-43f852b5-8c30-462c-aa4d-362111b4a9db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:50.506603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:50.508928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:50.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:50.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:50.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:50.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:50.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:50.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:50.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:50.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:50.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 469/948] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:50.514891 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:50.626011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8f883-9dcc-494c-af00-462a1c94550d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:50.654432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60549-9e3b-478f-a243-19c8f9a01dc1 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:50.654675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60549-9e3b-478f-a243-19c8f9a01dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:50.654882 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60549-9e3b-478f-a243-19c8f9a01dc1 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:50.660749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 470/949] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:50.663176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c468d-f174-48a7-b8a8-8a6b44d8e935 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:50.663494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c468d-f174-48a7-b8a8-8a6b44d8e935 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:50.663815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c468d-f174-48a7-b8a8-8a6b44d8e935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:50.669072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782859f-7ff7-4658-9506-9ae18ca6c23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:50.669690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782859f-7ff7-4658-9506-9ae18ca6c23f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:50.670024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782859f-7ff7-4658-9506-9ae18ca6c23f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:50.670256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782859f-7ff7-4658-9506-9ae18ca6c23f None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:50.670942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782859f-7ff7-4658-9506-9ae18ca6c23f None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:50.671243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782859f-7ff7-4658-9506-9ae18ca6c23f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:50.672895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782859f-7ff7-4658-9506-9ae18ca6c23f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:50.689881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f85c468d-f174-48a7-b8a8-8a6b44d8e935 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:50.691441 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f85c468d-f174-48a7-b8a8-8a6b44d8e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:50.694868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 480/950] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:50.771977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 471/951] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:51.050074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782859f-7ff7-4658-9506-9ae18ca6c23f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:51.106402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:51.156223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782859f-7ff7-4658-9506-9ae18ca6c23f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:51.192315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0731e-0fb3-4c69-93d6-328f8ee5a409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:51.192985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0731e-0fb3-4c69-93d6-328f8ee5a409 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.193267 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0731e-0fb3-4c69-93d6-328f8ee5a409 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.193541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0731e-0fb3-4c69-93d6-328f8ee5a409 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.205145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0731e-0fb3-4c69-93d6-328f8ee5a409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-48c0731e-0fb3-4c69-93d6-328f8ee5a409 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:12:51.210343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:51.211604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:12:51.212475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:51.216748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 481/952] 10.222.0.84 () {56 vars in 1120 bytes} [Wed Aug 24 11:12:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:51.218127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c468d-f174-48a7-b8a8-8a6b44d8e935 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:51.233153 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:51.327841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c468d-f174-48a7-b8a8-8a6b44d8e935 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:51.363365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d096244-f9ff-4944-96ce-91d371cadc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:51.363968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096244-f9ff-4944-96ce-91d371cadc06 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.364243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096244-f9ff-4944-96ce-91d371cadc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.364512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096244-f9ff-4944-96ce-91d371cadc06 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.365204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d096244-f9ff-4944-96ce-91d371cadc06 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:51.365503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d096244-f9ff-4944-96ce-91d371cadc06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:51.366168 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd542c3-06f6-4f48-a787-fcfc278c1f4b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.367532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd542c3-06f6-4f48-a787-fcfc278c1f4b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.367637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d096244-f9ff-4944-96ce-91d371cadc06 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:51.368102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd542c3-06f6-4f48-a787-fcfc278c1f4b None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.369981 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 482/953] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:12:51.375484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb510ef-e753-41b0-a8ca-05347263b525 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.376065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb510ef-e753-41b0-a8ca-05347263b525 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.376549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb510ef-e753-41b0-a8ca-05347263b525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.376759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 472/954] 10.222.0.84 () {56 vars in 1115 bytes} [Wed Aug 24 11:12:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:51.382111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d096244-f9ff-4944-96ce-91d371cadc06 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:51.399976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-afb510ef-e753-41b0-a8ca-05347263b525 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:51.402088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-afb510ef-e753-41b0-a8ca-05347263b525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:51.463455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:51.486012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 483/955] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:51.487850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a924b-07b0-4205-b0cd-cb02e61abd53 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.488343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a924b-07b0-4205-b0cd-cb02e61abd53 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.488841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a924b-07b0-4205-b0cd-cb02e61abd53 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.490597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 484/956] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:51.491056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d096244-f9ff-4944-96ce-91d371cadc06 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:51.496184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb349dd-3006-4e46-b997-74b913e00c96 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.496721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb349dd-3006-4e46-b997-74b913e00c96 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.497259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb349dd-3006-4e46-b997-74b913e00c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.516494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-2fb349dd-3006-4e46-b997-74b913e00c96 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:51.518038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2fb349dd-3006-4e46-b997-74b913e00c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:51.527045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c4d6e-3cc9-47ee-b787-c66122641524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:51.527675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c4d6e-3cc9-47ee-b787-c66122641524 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.527951 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c4d6e-3cc9-47ee-b787-c66122641524 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.528221 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c4d6e-3cc9-47ee-b787-c66122641524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.539586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c4d6e-3cc9-47ee-b787-c66122641524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:51.541047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c4d6e-3cc9-47ee-b787-c66122641524 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-4b2c4d6e-3cc9-47ee-b787-c66122641524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:51.555353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:51.556370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:51.562445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:51.562445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:51.562445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:51.562445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 473/957] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:51.564180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c4d6e-3cc9-47ee-b787-c66122641524 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:51.565806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:51.612423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 485/958] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:51.676471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c4d6e-3cc9-47ee-b787-c66122641524 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:51.703581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:51.713023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6adb1-3837-4521-be82-dbaeb954556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:51.720467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6adb1-3837-4521-be82-dbaeb954556a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.720467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6adb1-3837-4521-be82-dbaeb954556a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.720467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6adb1-3837-4521-be82-dbaeb954556a None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.720467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6adb1-3837-4521-be82-dbaeb954556a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:51.720467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6adb1-3837-4521-be82-dbaeb954556a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:51.720467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6adb1-3837-4521-be82-dbaeb954556a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:51.728598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 474/959] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:51.734749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb349dd-3006-4e46-b997-74b913e00c96 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:51.790203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6adb1-3837-4521-be82-dbaeb954556a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:51.862086 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb349dd-3006-4e46-b997-74b913e00c96 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:51.900141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6adb1-3837-4521-be82-dbaeb954556a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:51.909315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff801ecf-37dd-495d-98a3-3500decf96f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:51.910251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801ecf-37dd-495d-98a3-3500decf96f9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.910251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801ecf-37dd-495d-98a3-3500decf96f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.910469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801ecf-37dd-495d-98a3-3500decf96f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.918500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff801ecf-37dd-495d-98a3-3500decf96f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:51.919928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff801ecf-37dd-495d-98a3-3500decf96f9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-ff801ecf-37dd-495d-98a3-3500decf96f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:51.931542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:51.932697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 486/960] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:12:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:51.933902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff801ecf-37dd-495d-98a3-3500decf96f9 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:51.940423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381056-4f84-4a5c-8bbe-dcd6db3444d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:51.941087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381056-4f84-4a5c-8bbe-dcd6db3444d1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:51.941366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381056-4f84-4a5c-8bbe-dcd6db3444d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:51.941635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55381056-4f84-4a5c-8bbe-dcd6db3444d1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:51.942218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55381056-4f84-4a5c-8bbe-dcd6db3444d1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:51.943563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55381056-4f84-4a5c-8bbe-dcd6db3444d1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-55381056-4f84-4a5c-8bbe-dcd6db3444d1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:51.952695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:51.953772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:51.954603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 475/961] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:51.955406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381056-4f84-4a5c-8bbe-dcd6db3444d1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:51.973402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:52.010012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:52.046701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff801ecf-37dd-495d-98a3-3500decf96f9 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.066205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55381056-4f84-4a5c-8bbe-dcd6db3444d1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.084201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa86ff-6cc2-4057-a876-f8eea807ae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:52.084849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa86ff-6cc2-4057-a876-f8eea807ae6b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.085127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa86ff-6cc2-4057-a876-f8eea807ae6b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.085990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa86ff-6cc2-4057-a876-f8eea807ae6b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.086965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa86ff-6cc2-4057-a876-f8eea807ae6b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:52.088657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa86ff-6cc2-4057-a876-f8eea807ae6b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-baaa86ff-6cc2-4057-a876-f8eea807ae6b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:52.099374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:52.100380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:52.102260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 487/962] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:52.103065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa86ff-6cc2-4057-a876-f8eea807ae6b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:52.106630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef3ef3-6947-4ee0-97ad-df3dbfd99b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:52.107254 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef3ef3-6947-4ee0-97ad-df3dbfd99b92 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.107530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef3ef3-6947-4ee0-97ad-df3dbfd99b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.107807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef3ef3-6947-4ee0-97ad-df3dbfd99b92 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.108813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef3ef3-6947-4ee0-97ad-df3dbfd99b92 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:52.109115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef3ef3-6947-4ee0-97ad-df3dbfd99b92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:52.110540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef3ef3-6947-4ee0-97ad-df3dbfd99b92 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:52.129422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 476/963] 10.222.0.84 () {56 vars in 1149 bytes} [Wed Aug 24 11:12:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:52.130233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef3ef3-6947-4ee0-97ad-df3dbfd99b92 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:52.208622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:52.216101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa86ff-6cc2-4057-a876-f8eea807ae6b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.236531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef3ef3-6947-4ee0-97ad-df3dbfd99b92 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.247145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:52.252062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3692ba6-79a2-4334-a46b-79675da15afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:52.252672 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3692ba6-79a2-4334-a46b-79675da15afe None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.252986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3692ba6-79a2-4334-a46b-79675da15afe None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.253255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3692ba6-79a2-4334-a46b-79675da15afe None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.253877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3692ba6-79a2-4334-a46b-79675da15afe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:52.254176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3692ba6-79a2-4334-a46b-79675da15afe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:52.255534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3692ba6-79a2-4334-a46b-79675da15afe None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:52.260960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 488/964] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:52.261745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3692ba6-79a2-4334-a46b-79675da15afe None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:52.280424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a1f4f-6c9b-421e-9e2f-615025f3e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:52.281453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a1f4f-6c9b-421e-9e2f-615025f3e4ff None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.282046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a1f4f-6c9b-421e-9e2f-615025f3e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.282386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a1f4f-6c9b-421e-9e2f-615025f3e4ff None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.283023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a1f4f-6c9b-421e-9e2f-615025f3e4ff None admin] RBAC: Adding query filter params (name=compute) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:52.283323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a1f4f-6c9b-421e-9e2f-615025f3e4ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:52.284854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a1f4f-6c9b-421e-9e2f-615025f3e4ff None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:52.294489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 477/965] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:52.294489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a1f4f-6c9b-421e-9e2f-615025f3e4ff None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:52.374113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3692ba6-79a2-4334-a46b-79675da15afe None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.402015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a1f4f-6c9b-421e-9e2f-615025f3e4ff None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.411003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e484c-8dc3-49a7-a274-9071f27e7079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:52.411630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e484c-8dc3-49a7-a274-9071f27e7079 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.411911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e484c-8dc3-49a7-a274-9071f27e7079 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.412181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e484c-8dc3-49a7-a274-9071f27e7079 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.437322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993e532-4832-47f7-afa9-ca1b383bd2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:52.438050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e532-4832-47f7-afa9-ca1b383bd2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.438328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e532-4832-47f7-afa9-ca1b383bd2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.438617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e532-4832-47f7-afa9-ca1b383bd2e3 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.439241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993e532-4832-47f7-afa9-ca1b383bd2e3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:52.439543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993e532-4832-47f7-afa9-ca1b383bd2e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:52.440908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993e532-4832-47f7-afa9-ca1b383bd2e3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:52.442213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:52.447031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 478/966] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:52.447813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993e532-4832-47f7-afa9-ca1b383bd2e3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:52.490714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:52.491312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e484c-8dc3-49a7-a274-9071f27e7079 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:52.492502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e484c-8dc3-49a7-a274-9071f27e7079 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:52.564901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993e532-4832-47f7-afa9-ca1b383bd2e3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.600687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 489/967] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:12:52 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:12:52.602157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e484c-8dc3-49a7-a274-9071f27e7079 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:52.608809 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6baac-4d97-46f3-860f-3ec0d94c676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:52.609727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6baac-4d97-46f3-860f-3ec0d94c676a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.610230 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6baac-4d97-46f3-860f-3ec0d94c676a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.611098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6baac-4d97-46f3-860f-3ec0d94c676a None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.611931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6baac-4d97-46f3-860f-3ec0d94c676a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:52.612814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6baac-4d97-46f3-860f-3ec0d94c676a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:52.614276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6baac-4d97-46f3-860f-3ec0d94c676a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:52.620006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 479/968] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:52.620778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6baac-4d97-46f3-860f-3ec0d94c676a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:52.688581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:52.739110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e484c-8dc3-49a7-a274-9071f27e7079 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.770670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:52.774968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6baac-4d97-46f3-860f-3ec0d94c676a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.783663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0680830-92bc-4bdc-9289-3b0a7c474a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:52.785647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0680830-92bc-4bdc-9289-3b0a7c474a28 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.785647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0680830-92bc-4bdc-9289-3b0a7c474a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.785647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0680830-92bc-4bdc-9289-3b0a7c474a28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.785647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0680830-92bc-4bdc-9289-3b0a7c474a28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:52.786798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0680830-92bc-4bdc-9289-3b0a7c474a28 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:52.812105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71719f75-38f7-404b-a747-50e6d960d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:52.812754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71719f75-38f7-404b-a747-50e6d960d655 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.813067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71719f75-38f7-404b-a747-50e6d960d655 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.813335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71719f75-38f7-404b-a747-50e6d960d655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.824518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71719f75-38f7-404b-a747-50e6d960d655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:52.826050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71719f75-38f7-404b-a747-50e6d960d655 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-71719f75-38f7-404b-a747-50e6d960d655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:12:52.841185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:52.842191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:12:52.843112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:12:52.843112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:12:52.843112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:52.843112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 480/969] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:52.843752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2f592-2211-4a0a-8390-6657d191510a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.843913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2f592-2211-4a0a-8390-6657d191510a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.844305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2f592-2211-4a0a-8390-6657d191510a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.845683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 481/970] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:52.849792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2f592-2211-4a0a-8390-6657d191510a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:52.933268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 490/971] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:12:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:52.934970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5400ad6-0b0d-40dd-968c-8b2ab6c16fb5 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:52.935610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5400ad6-0b0d-40dd-968c-8b2ab6c16fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:52.936017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5400ad6-0b0d-40dd-968c-8b2ab6c16fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:52.964786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2f592-2211-4a0a-8390-6657d191510a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:52.969246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e5400ad6-0b0d-40dd-968c-8b2ab6c16fb5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:52.970621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e5400ad6-0b0d-40dd-968c-8b2ab6c16fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:53.006179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b98cf0-42e1-4016-8253-b19bc42de277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:53.007070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b98cf0-42e1-4016-8253-b19bc42de277 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:53.008247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b98cf0-42e1-4016-8253-b19bc42de277 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:53.008619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b98cf0-42e1-4016-8253-b19bc42de277 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:53.009479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b98cf0-42e1-4016-8253-b19bc42de277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:53.009974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b98cf0-42e1-4016-8253-b19bc42de277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:53.011578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b98cf0-42e1-4016-8253-b19bc42de277 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:53.026717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 482/972] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:53.032680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b98cf0-42e1-4016-8253-b19bc42de277 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:53.048993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:53.068968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 491/973] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:53.156215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b98cf0-42e1-4016-8253-b19bc42de277 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:53.194170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803e9fe-62f2-4728-815f-d5e064b61464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:53.194831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803e9fe-62f2-4728-815f-d5e064b61464 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:53.195702 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803e9fe-62f2-4728-815f-d5e064b61464 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:53.195982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803e9fe-62f2-4728-815f-d5e064b61464 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:53.203071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803e9fe-62f2-4728-815f-d5e064b61464 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:53.204589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803e9fe-62f2-4728-815f-d5e064b61464 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f803e9fe-62f2-4728-815f-d5e064b61464 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:12:53.216828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:53.217707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:53.218580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:12:53.218580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:12:53.218580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:12:53.218580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:12:53.218580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:12:53.218580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:12:53.218580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:12:53.218580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:53.218580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 483/974] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:53.225338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5400ad6-0b0d-40dd-968c-8b2ab6c16fb5 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:53.276531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:53.331029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5400ad6-0b0d-40dd-968c-8b2ab6c16fb5 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:53.353976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803e9fe-62f2-4728-815f-d5e064b61464 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:53.374198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13c3a6-0a98-413e-a8de-6ce63f80402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:53.374845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13c3a6-0a98-413e-a8de-6ce63f80402e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:53.375065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13c3a6-0a98-413e-a8de-6ce63f80402e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:53.375329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13c3a6-0a98-413e-a8de-6ce63f80402e None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:53.376021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13c3a6-0a98-413e-a8de-6ce63f80402e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:53.376319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13c3a6-0a98-413e-a8de-6ce63f80402e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:53.377760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13c3a6-0a98-413e-a8de-6ce63f80402e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:53.391592 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 492/975] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:53.397448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13c3a6-0a98-413e-a8de-6ce63f80402e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:53.444357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:53.492061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803e9fe-62f2-4728-815f-d5e064b61464 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:53.516809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13c3a6-0a98-413e-a8de-6ce63f80402e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:53.526092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:53.531806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:53.532608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:53.533086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:53.535268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:53.535268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:53.536650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:53.540219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING keystone.server.flask.common [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 11:12:53.557469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333a93-9be1-46e1-afc0-ba20300305d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:53.558155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333a93-9be1-46e1-afc0-ba20300305d7 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:53.558497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333a93-9be1-46e1-afc0-ba20300305d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:53.558832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66333a93-9be1-46e1-afc0-ba20300305d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:53.559609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333a93-9be1-46e1-afc0-ba20300305d7 None admin] RBAC: Adding query filter params (role.id=b2da7bb417d94f45a2a4e8a2befa0988, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=164f3bcc6363445da191bd1b78499a53) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:53.559994 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333a93-9be1-46e1-afc0-ba20300305d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:53.561512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66333a93-9be1-46e1-afc0-ba20300305d7 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:53.567731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 493/976] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:12:53 2022] GET /identity/v3/role_assignments?user.id=164f3bcc6363445da191bd1b78499a53&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=b2da7bb417d94f45a2a4e8a2befa0988 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:53.597452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.sql.core [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 11:12:53.597452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 11:12:53.597452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [parameters: {'user_id': '88f59276fb4143a4be8a7cd8bbced85e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 11:12:53.597452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81706) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result.read() Aug 24 11:12:53.607450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:12:53.608677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application yield resource Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application next(self.gen) Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise self.value Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application session.commit() Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 11:12:53.609642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.session.flush() Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application compat.raise_( Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 11:12:53.610544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 11:12:53.612144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application conn.query(q) Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application result.read() Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 11:12:53.613715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [parameters: {'user_id': '88f59276fb4143a4be8a7cd8bbced85e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:53.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 11:12:53.618384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 11:12:53.620587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 11:12:53.620587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 11:12:53.620587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 11:12:53.620587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 11:12:53.620587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 11:12:53.620587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:53.620587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 484/977] 10.222.0.84 () {60 vars in 1131 bytes} [Wed Aug 24 11:12:53 2022] POST /identity/v3/users => generated 160 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:53.621527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333a93-9be1-46e1-afc0-ba20300305d7 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:53.740965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66333a93-9be1-46e1-afc0-ba20300305d7 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:53.779027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472aff4a-4b15-4fbf-a458-63996db26ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:53.779992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aff4a-4b15-4fbf-a458-63996db26ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:53.780411 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aff4a-4b15-4fbf-a458-63996db26ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:53.780868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aff4a-4b15-4fbf-a458-63996db26ef4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:53.789446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472aff4a-4b15-4fbf-a458-63996db26ef4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:53.791217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472aff4a-4b15-4fbf-a458-63996db26ef4 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-472aff4a-4b15-4fbf-a458-63996db26ef4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:12:53.802715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:53.803794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:53.805777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 494/978] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:12:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:53.811708 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:53.918047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099bc5b-e2f4-4045-88ed-a5fcabb18a94 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:53.948405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:53.954162 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ea7a2-6217-4a1d-8156-44fd88048ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:53.955084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea7a2-6217-4a1d-8156-44fd88048ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:53.955360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea7a2-6217-4a1d-8156-44fd88048ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:53.955627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea7a2-6217-4a1d-8156-44fd88048ac4 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:53.956322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ea7a2-6217-4a1d-8156-44fd88048ac4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:53.956619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ea7a2-6217-4a1d-8156-44fd88048ac4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:53.958261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ea7a2-6217-4a1d-8156-44fd88048ac4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:53.998195 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 485/979] 10.222.0.84 () {56 vars in 1101 bytes} [Wed Aug 24 11:12:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:54.607420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3977-8f34-4337-9049-71cab993bbf7 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:54.608227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3977-8f34-4337-9049-71cab993bbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:54.608227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b3977-8f34-4337-9049-71cab993bbf7 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:54.609602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 495/980] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:54.614962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ee803-2b25-405a-ab05-0cd860e46e23 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:54.615349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ee803-2b25-405a-ab05-0cd860e46e23 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:54.615510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ee803-2b25-405a-ab05-0cd860e46e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:54.634170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e20ee803-2b25-405a-ab05-0cd860e46e23 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:54.635760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e20ee803-2b25-405a-ab05-0cd860e46e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:54.709719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 486/981] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:54.982415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3977-8f34-4337-9049-71cab993bbf7 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:55.010416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f08689-36e3-4ba5-ac97-63bc965ec51f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:55.010984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f08689-36e3-4ba5-ac97-63bc965ec51f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:55.011270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f08689-36e3-4ba5-ac97-63bc965ec51f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:55.013158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 487/982] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:55.015058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:55.018347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40600d91-eb24-4193-ae8a-9374f34475df None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:55.018633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40600d91-eb24-4193-ae8a-9374f34475df None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:55.018905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40600d91-eb24-4193-ae8a-9374f34475df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:55.037624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-40600d91-eb24-4193-ae8a-9374f34475df None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:55.039154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-40600d91-eb24-4193-ae8a-9374f34475df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:55.097184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b3977-8f34-4337-9049-71cab993bbf7 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:55.108714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 488/983] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:55.138269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0ccc3-6cc7-46c4-95ab-e3331af71a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:55.138866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0ccc3-6cc7-46c4-95ab-e3331af71a33 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:55.139126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0ccc3-6cc7-46c4-95ab-e3331af71a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:55.139389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0ccc3-6cc7-46c4-95ab-e3331af71a33 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:55.140029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0ccc3-6cc7-46c4-95ab-e3331af71a33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:55.141423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0ccc3-6cc7-46c4-95ab-e3331af71a33 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:55.227181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 496/984] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:12:54 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:12:55.383691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40600d91-eb24-4193-ae8a-9374f34475df None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:55.434769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:55.486903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40600d91-eb24-4193-ae8a-9374f34475df None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:55.528641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86779d-dff2-4607-8816-ebaa534d5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:55.529299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86779d-dff2-4607-8816-ebaa534d5145 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:55.529571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86779d-dff2-4607-8816-ebaa534d5145 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:55.529830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86779d-dff2-4607-8816-ebaa534d5145 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:55.530405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86779d-dff2-4607-8816-ebaa534d5145 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:55.531752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86779d-dff2-4607-8816-ebaa534d5145 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5c86779d-dff2-4607-8816-ebaa534d5145 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:55.541539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:55.542552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:55.543382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:12:55.544877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 489/985] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:55.549025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0ccc3-6cc7-46c4-95ab-e3331af71a33 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:55.627993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a05986-2ce7-4152-92a0-8b7a167af692 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:55.628551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a05986-2ce7-4152-92a0-8b7a167af692 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:55.628551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a05986-2ce7-4152-92a0-8b7a167af692 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:55.630016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 490/986] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:55.634904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8093a6-a562-4c17-b83b-1540a582ec4d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:55.635124 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8093a6-a562-4c17-b83b-1540a582ec4d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:55.635392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8093a6-a562-4c17-b83b-1540a582ec4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:55.641980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:55.653648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ec8093a6-a562-4c17-b83b-1540a582ec4d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:55.654854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ec8093a6-a562-4c17-b83b-1540a582ec4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:55.664940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0ccc3-6cc7-46c4-95ab-e3331af71a33 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:55.703695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c32920-0d67-420c-acec-131b8f230883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:55.704609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c32920-0d67-420c-acec-131b8f230883 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:55.704927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c32920-0d67-420c-acec-131b8f230883 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:55.705196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c32920-0d67-420c-acec-131b8f230883 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:55.705821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c32920-0d67-420c-acec-131b8f230883 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:55.706117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c32920-0d67-420c-acec-131b8f230883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:55.707470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c32920-0d67-420c-acec-131b8f230883 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:55.717820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 497/987] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:55.719333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c32920-0d67-420c-acec-131b8f230883 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:55.737836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 491/988] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:55.821027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c32920-0d67-420c-acec-131b8f230883 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:55.854469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eb25a-293b-4f13-ad88-0676d2e4a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:55.855069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eb25a-293b-4f13-ad88-0676d2e4a3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:55.855350 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eb25a-293b-4f13-ad88-0676d2e4a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:55.855609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eb25a-293b-4f13-ad88-0676d2e4a3f2 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:55.856228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eb25a-293b-4f13-ad88-0676d2e4a3f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:55.856512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eb25a-293b-4f13-ad88-0676d2e4a3f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:55.857927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eb25a-293b-4f13-ad88-0676d2e4a3f2 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:55.867837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 498/989] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:55.869359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8093a6-a562-4c17-b83b-1540a582ec4d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:55.909436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:55.968674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8093a6-a562-4c17-b83b-1540a582ec4d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:56.002778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec9764-8587-48a7-ad60-5ba8c4d5d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:56.003385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec9764-8587-48a7-ad60-5ba8c4d5d665 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:56.003663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eb25a-293b-4f13-ad88-0676d2e4a3f2 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:56.003794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec9764-8587-48a7-ad60-5ba8c4d5d665 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:56.003952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec9764-8587-48a7-ad60-5ba8c4d5d665 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:56.004622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec9764-8587-48a7-ad60-5ba8c4d5d665 None admin] RBAC: Adding query filter params (interface=admin, service_id=f7f4e3aa6e2f4bc483b019cecb2f82a1, region_id=RegionOne) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:56.004955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec9764-8587-48a7-ad60-5ba8c4d5d665 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:56.006309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec9764-8587-48a7-ad60-5ba8c4d5d665 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:56.014565 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 492/990] 10.222.0.84 () {56 vars in 1249 bytes} [Wed Aug 24 11:12:55 2022] GET /identity/v3/endpoints?service_id=f7f4e3aa6e2f4bc483b019cecb2f82a1&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:56.022806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec9764-8587-48a7-ad60-5ba8c4d5d665 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:56.040162 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:56.123636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eb25a-293b-4f13-ad88-0676d2e4a3f2 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:56.133590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec9764-8587-48a7-ad60-5ba8c4d5d665 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:56.135122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:56.161384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157af2c-ee93-4efb-a6eb-dfd99492260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:56.161929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157af2c-ee93-4efb-a6eb-dfd99492260a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:56.162204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157af2c-ee93-4efb-a6eb-dfd99492260a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:56.162471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157af2c-ee93-4efb-a6eb-dfd99492260a None admin] PATH_INFO: `/v3/services/image` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:56.163044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157af2c-ee93-4efb-a6eb-dfd99492260a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:56.164511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157af2c-ee93-4efb-a6eb-dfd99492260a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:56.170448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeca19-6cac-4a03-bf15-ccbc9bdecc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:56.171048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeca19-6cac-4a03-bf15-ccbc9bdecc4c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:56.171320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeca19-6cac-4a03-bf15-ccbc9bdecc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:56.171587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeca19-6cac-4a03-bf15-ccbc9bdecc4c None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:56.172184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeeca19-6cac-4a03-bf15-ccbc9bdecc4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8157af2c-ee93-4efb-a6eb-dfd99492260a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:56.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:56.173872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:12:56.174711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:56.175450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeeca19-6cac-4a03-bf15-ccbc9bdecc4c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:56.178979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 499/991] 10.222.0.84 () {56 vars in 1105 bytes} [Wed Aug 24 11:12:56 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:56.179957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157af2c-ee93-4efb-a6eb-dfd99492260a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:56.193165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 493/992] 10.222.0.84 () {56 vars in 1087 bytes} [Wed Aug 24 11:12:56 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:12:56.271077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:56.290829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157af2c-ee93-4efb-a6eb-dfd99492260a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:56.331441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ff414-6555-4fc2-8f24-301278a2ddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:56.332094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ff414-6555-4fc2-8f24-301278a2ddb9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:56.332371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ff414-6555-4fc2-8f24-301278a2ddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:56.332636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ff414-6555-4fc2-8f24-301278a2ddb9 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:56.333301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ff414-6555-4fc2-8f24-301278a2ddb9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:56.333599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ff414-6555-4fc2-8f24-301278a2ddb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:56.335003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ff414-6555-4fc2-8f24-301278a2ddb9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:56.340559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 500/993] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:56.346478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeca19-6cac-4a03-bf15-ccbc9bdecc4c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:56.452722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeca19-6cac-4a03-bf15-ccbc9bdecc4c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:56.488243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad64bed-3981-4d6b-b50f-b13b0eb53631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:56.488871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad64bed-3981-4d6b-b50f-b13b0eb53631 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:56.489148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad64bed-3981-4d6b-b50f-b13b0eb53631 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:56.489421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad64bed-3981-4d6b-b50f-b13b0eb53631 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:56.490048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad64bed-3981-4d6b-b50f-b13b0eb53631 None admin] RBAC: Adding query filter params (type=image) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:56.490346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad64bed-3981-4d6b-b50f-b13b0eb53631 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:56.491751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad64bed-3981-4d6b-b50f-b13b0eb53631 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:56.496596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 494/994] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:12:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:57.171436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c88560-fcf3-43a4-b6d0-df69d0c816e8 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:57.171963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c88560-fcf3-43a4-b6d0-df69d0c816e8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:57.171963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c88560-fcf3-43a4-b6d0-df69d0c816e8 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:57.173527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 501/995] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:57.178707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6178b-e37e-442b-9cfb-acd632ae5f38 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:57.179217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6178b-e37e-442b-9cfb-acd632ae5f38 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:57.179295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6178b-e37e-442b-9cfb-acd632ae5f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:57.197604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a2a6178b-e37e-442b-9cfb-acd632ae5f38 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:57.198805 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a2a6178b-e37e-442b-9cfb-acd632ae5f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:57.223591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:57.268897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 495/996] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:57.567593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c88560-fcf3-43a4-b6d0-df69d0c816e8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:57.679581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c88560-fcf3-43a4-b6d0-df69d0c816e8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:57.713694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926d106-7515-4dce-ace3-5b46518cec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:57.714314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926d106-7515-4dce-ace3-5b46518cec63 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:57.714586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926d106-7515-4dce-ace3-5b46518cec63 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:57.714850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926d106-7515-4dce-ace3-5b46518cec63 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:57.715415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926d106-7515-4dce-ace3-5b46518cec63 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:57.716768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926d106-7515-4dce-ace3-5b46518cec63 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8926d106-7515-4dce-ace3-5b46518cec63 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:57.725497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:57.726699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:57.729731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 502/997] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:12:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:12:57.733252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6178b-e37e-442b-9cfb-acd632ae5f38 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:57.832952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6178b-e37e-442b-9cfb-acd632ae5f38 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:57.868038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e097d-453c-4ee3-af24-902a7f7d6d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:57.868644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e097d-453c-4ee3-af24-902a7f7d6d83 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:57.868952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e097d-453c-4ee3-af24-902a7f7d6d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:57.869309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e097d-453c-4ee3-af24-902a7f7d6d83 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:57.869937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e097d-453c-4ee3-af24-902a7f7d6d83 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:57.870236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e097d-453c-4ee3-af24-902a7f7d6d83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:57.871588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e097d-453c-4ee3-af24-902a7f7d6d83 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:57.881115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 496/998] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:57.882987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926d106-7515-4dce-ace3-5b46518cec63 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:57.895768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:57.984924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926d106-7515-4dce-ace3-5b46518cec63 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:58.020015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418689f-31a8-431c-932d-e28f3e6c38f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:58.020949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418689f-31a8-431c-932d-e28f3e6c38f4 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.021366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418689f-31a8-431c-932d-e28f3e6c38f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.021773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418689f-31a8-431c-932d-e28f3e6c38f4 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.022532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418689f-31a8-431c-932d-e28f3e6c38f4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:58.023025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418689f-31a8-431c-932d-e28f3e6c38f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:58.024539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418689f-31a8-431c-932d-e28f3e6c38f4 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:58.034309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 503/999] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:12:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:58.035572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e097d-453c-4ee3-af24-902a7f7d6d83 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:58.063073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:58.122140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b04e1-c83a-4446-b4ce-a565bbb1f88b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.122442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b04e1-c83a-4446-b4ce-a565bbb1f88b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.122616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b04e1-c83a-4446-b4ce-a565bbb1f88b None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.123942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 504/1000] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:12:58.128759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594656e-4a21-41cf-bbc7-a3726f45092a None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.129071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594656e-4a21-41cf-bbc7-a3726f45092a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.129343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594656e-4a21-41cf-bbc7-a3726f45092a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.142452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e097d-453c-4ee3-af24-902a7f7d6d83 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:58.147206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b594656e-4a21-41cf-bbc7-a3726f45092a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:58.148980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b594656e-4a21-41cf-bbc7-a3726f45092a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:58.178988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c667a-5bf4-4630-b4bd-a895b31e7da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:58.179829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c667a-5bf4-4630-b4bd-a895b31e7da3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.180105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c667a-5bf4-4630-b4bd-a895b31e7da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.180376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c667a-5bf4-4630-b4bd-a895b31e7da3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.181087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c667a-5bf4-4630-b4bd-a895b31e7da3 None admin] RBAC: Adding query filter params (interface=public, service_id=1a1230292b654dfbb216f01ae8907176, region_id=RegionOne) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:58.181403 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c667a-5bf4-4630-b4bd-a895b31e7da3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:58.182839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c667a-5bf4-4630-b4bd-a895b31e7da3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:58.187969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 497/1001] 10.222.0.84 () {56 vars in 1251 bytes} [Wed Aug 24 11:12:58 2022] GET /identity/v3/endpoints?service_id=1a1230292b654dfbb216f01ae8907176&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:58.193440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c667a-5bf4-4630-b4bd-a895b31e7da3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:58.228908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 505/1002] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:12:58.283528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:58.296417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c667a-5bf4-4630-b4bd-a895b31e7da3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:58.330931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928f4c8-2892-47de-9eae-e8b1ba95fb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:58.331529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928f4c8-2892-47de-9eae-e8b1ba95fb1f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.331800 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928f4c8-2892-47de-9eae-e8b1ba95fb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.332065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928f4c8-2892-47de-9eae-e8b1ba95fb1f None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.332650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928f4c8-2892-47de-9eae-e8b1ba95fb1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:58.334023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928f4c8-2892-47de-9eae-e8b1ba95fb1f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:58.345041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 498/1003] 10.222.0.84 () {56 vars in 1087 bytes} [Wed Aug 24 11:12:58 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:12:58.408173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc9496-1ed8-4433-bfc7-c7affef83814 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.408379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc9496-1ed8-4433-bfc7-c7affef83814 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.408623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc9496-1ed8-4433-bfc7-c7affef83814 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.410026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 506/1004] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:12:58.416771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e26b2-5af9-47f9-8550-d199e0f62641 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.417084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e26b2-5af9-47f9-8550-d199e0f62641 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.417354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e26b2-5af9-47f9-8550-d199e0f62641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.436858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-515e26b2-5af9-47f9-8550-d199e0f62641 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:12:58.438062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-515e26b2-5af9-47f9-8550-d199e0f62641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:12:58.499112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc9496-1ed8-4433-bfc7-c7affef83814 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:58.503547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:58.512149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 499/1005] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:12:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:12:58.603181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc9496-1ed8-4433-bfc7-c7affef83814 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:58.637463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b4d09-95dc-48f5-b490-07a5c6735bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:58.638091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b4d09-95dc-48f5-b490-07a5c6735bed None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.638366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b4d09-95dc-48f5-b490-07a5c6735bed None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.638635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b4d09-95dc-48f5-b490-07a5c6735bed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.639212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b4d09-95dc-48f5-b490-07a5c6735bed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:58.640570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b4d09-95dc-48f5-b490-07a5c6735bed None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-476b4d09-95dc-48f5-b490-07a5c6735bed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:12:58.650725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:12:58.651741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:12:58.652595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 507/1006] 10.222.0.84 () {56 vars in 1126 bytes} [Wed Aug 24 11:12:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:12:58.660044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e26b2-5af9-47f9-8550-d199e0f62641 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:58.745166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:58.763043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e26b2-5af9-47f9-8550-d199e0f62641 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:58.780502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b4d09-95dc-48f5-b490-07a5c6735bed None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:58.801265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c39f8-3727-4313-8323-1ccc69061dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:58.802456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c39f8-3727-4313-8323-1ccc69061dcc None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.802456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c39f8-3727-4313-8323-1ccc69061dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.802456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c39f8-3727-4313-8323-1ccc69061dcc None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.803015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c39f8-3727-4313-8323-1ccc69061dcc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:58.803310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c39f8-3727-4313-8323-1ccc69061dcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:58.805373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c39f8-3727-4313-8323-1ccc69061dcc None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:58.810832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 500/1007] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:12:58.816225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c39f8-3727-4313-8323-1ccc69061dcc None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:58.862259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:58.901178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b4d09-95dc-48f5-b490-07a5c6735bed None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:58.931796 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c39f8-3727-4313-8323-1ccc69061dcc None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:58.937687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95389e-49fb-4f93-8773-4f40e71280ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:58.938289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95389e-49fb-4f93-8773-4f40e71280ff None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.938564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95389e-49fb-4f93-8773-4f40e71280ff None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.938833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95389e-49fb-4f93-8773-4f40e71280ff None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.939583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95389e-49fb-4f93-8773-4f40e71280ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:58.941096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95389e-49fb-4f93-8773-4f40e71280ff None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:58.967394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed3273-39eb-4267-8c9d-5f7aeeb559be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:58.968000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed3273-39eb-4267-8c9d-5f7aeeb559be None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:58.968271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed3273-39eb-4267-8c9d-5f7aeeb559be None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:58.968911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed3273-39eb-4267-8c9d-5f7aeeb559be None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:58.969238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed3273-39eb-4267-8c9d-5f7aeeb559be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:12:58.969534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed3273-39eb-4267-8c9d-5f7aeeb559be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:58.970874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed3273-39eb-4267-8c9d-5f7aeeb559be None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:58.975665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 501/1008] 10.222.0.84 () {56 vars in 1122 bytes} [Wed Aug 24 11:12:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:12:58.981404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed3273-39eb-4267-8c9d-5f7aeeb559be None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:12:58.994292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:12:59.057044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 508/1009] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:12:58 2022] POST /identity/v3/services => generated 242 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:12:59.088523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed3273-39eb-4267-8c9d-5f7aeeb559be None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:12:59.141536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf33ddc-4670-4f56-ad12-154e867e2139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:12:59.142148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf33ddc-4670-4f56-ad12-154e867e2139 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:12:59.142421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf33ddc-4670-4f56-ad12-154e867e2139 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:12:59.142686 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf33ddc-4670-4f56-ad12-154e867e2139 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:12:59.143340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf33ddc-4670-4f56-ad12-154e867e2139 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:12:59.144713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf33ddc-4670-4f56-ad12-154e867e2139 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:12:59.255561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 502/1010] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:12:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:13:00.291525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a9429-93b5-4ce2-9c7d-077150981ea8 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:00.292052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a9429-93b5-4ce2-9c7d-077150981ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:00.292052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a9429-93b5-4ce2-9c7d-077150981ea8 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:00.293550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 509/1011] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:00.298780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d16e-fc5e-4d0a-91f0-f906a3bb379e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:00.299157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d16e-fc5e-4d0a-91f0-f906a3bb379e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:00.299316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61d16e-fc5e-4d0a-91f0-f906a3bb379e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:00.317745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4b61d16e-fc5e-4d0a-91f0-f906a3bb379e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:00.318919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4b61d16e-fc5e-4d0a-91f0-f906a3bb379e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:00.384834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:00.398236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 503/1012] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:00 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:00.668604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a9429-93b5-4ce2-9c7d-077150981ea8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:00.770443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a9429-93b5-4ce2-9c7d-077150981ea8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:00.796747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:00.805566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff79b1-2e55-4b89-8109-a5d081368c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:00.806188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff79b1-2e55-4b89-8109-a5d081368c34 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:00.809263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff79b1-2e55-4b89-8109-a5d081368c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:00.809263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff79b1-2e55-4b89-8109-a5d081368c34 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:00.809263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff79b1-2e55-4b89-8109-a5d081368c34 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:00.809263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff79b1-2e55-4b89-8109-a5d081368c34 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-5dff79b1-2e55-4b89-8109-a5d081368c34 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:00.817728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:00.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 11:13:00.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:00.822824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 510/1013] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:13:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:00.825512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d16e-fc5e-4d0a-91f0-f906a3bb379e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:00.950357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61d16e-fc5e-4d0a-91f0-f906a3bb379e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:00.984949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb7c49-3caa-474d-8c29-1051a6a313e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:00.985557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb7c49-3caa-474d-8c29-1051a6a313e1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:00.985834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb7c49-3caa-474d-8c29-1051a6a313e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:00.986103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb7c49-3caa-474d-8c29-1051a6a313e1 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:00.986724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb7c49-3caa-474d-8c29-1051a6a313e1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:00.987024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb7c49-3caa-474d-8c29-1051a6a313e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:00.988382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb7c49-3caa-474d-8c29-1051a6a313e1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:00.997875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 504/1014] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:13:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:00.999408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff79b1-2e55-4b89-8109-a5d081368c34 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:01.015707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e41eb-adee-4da9-b266-0eb95bc867f8 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.016014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e41eb-adee-4da9-b266-0eb95bc867f8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.016502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e41eb-adee-4da9-b266-0eb95bc867f8 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.018499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 505/1015] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:13:01.024257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ef89-43b9-45ce-95ad-bc34b02a5b39 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.024530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ef89-43b9-45ce-95ad-bc34b02a5b39 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.024812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ef89-43b9-45ce-95ad-bc34b02a5b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.043062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1257ef89-43b9-45ce-95ad-bc34b02a5b39 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:01.044232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1257ef89-43b9-45ce-95ad-bc34b02a5b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:01.099045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:01.110308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff79b1-2e55-4b89-8109-a5d081368c34 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:01.116962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 506/1016] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:01 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:13:01.147167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad496ac-aa8f-47ba-985f-c436bcce73dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:01.148310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad496ac-aa8f-47ba-985f-c436bcce73dd None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.148580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad496ac-aa8f-47ba-985f-c436bcce73dd None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.148935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad496ac-aa8f-47ba-985f-c436bcce73dd None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.149514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad496ac-aa8f-47ba-985f-c436bcce73dd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:01.149814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad496ac-aa8f-47ba-985f-c436bcce73dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:01.151170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad496ac-aa8f-47ba-985f-c436bcce73dd None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:01.160652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 511/1017] 10.222.0.84 () {56 vars in 1112 bytes} [Wed Aug 24 11:13:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:01.162307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ef89-43b9-45ce-95ad-bc34b02a5b39 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:01.173564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b712c-2866-4753-b103-8680b6442cd6 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.173851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b712c-2866-4753-b103-8680b6442cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.174124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b712c-2866-4753-b103-8680b6442cd6 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.175572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 512/1018] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:13:01.180614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7f916-b220-43f0-85c9-0e8469a3cb73 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.180995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7f916-b220-43f0-85c9-0e8469a3cb73 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.181331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7f916-b220-43f0-85c9-0e8469a3cb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.199680 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d5f7f916-b220-43f0-85c9-0e8469a3cb73 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:01.201245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d5f7f916-b220-43f0-85c9-0e8469a3cb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:01.260137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:01.276346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ef89-43b9-45ce-95ad-bc34b02a5b39 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:01.277405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 513/1019] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:01 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:13:01.311858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114e1e2-ac2a-4c6a-92d1-e2c386d1af32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:01.312549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114e1e2-ac2a-4c6a-92d1-e2c386d1af32 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.312758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114e1e2-ac2a-4c6a-92d1-e2c386d1af32 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.313013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114e1e2-ac2a-4c6a-92d1-e2c386d1af32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.313660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114e1e2-ac2a-4c6a-92d1-e2c386d1af32 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:01.315007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114e1e2-ac2a-4c6a-92d1-e2c386d1af32 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:01.392853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7f916-b220-43f0-85c9-0e8469a3cb73 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:01.404914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 507/1020] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:13:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:13:01.507536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7f916-b220-43f0-85c9-0e8469a3cb73 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:01.545263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114e1e2-ac2a-4c6a-92d1-e2c386d1af32 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:01.557523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cb33f-28c8-4e3b-8adb-d354c0633867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:01.558164 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cb33f-28c8-4e3b-8adb-d354c0633867 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.558440 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cb33f-28c8-4e3b-8adb-d354c0633867 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.558706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cb33f-28c8-4e3b-8adb-d354c0633867 None admin] PATH_INFO: `/v3/services/image` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.559282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cb33f-28c8-4e3b-8adb-d354c0633867 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:01.560642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cb33f-28c8-4e3b-8adb-d354c0633867 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-749cb33f-28c8-4e3b-8adb-d354c0633867 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:01.569773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:01.570983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:01.571823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 514/1021] 10.222.0.84 () {56 vars in 1105 bytes} [Wed Aug 24 11:13:01 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:13:01.577051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cb33f-28c8-4e3b-8adb-d354c0633867 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:01.599846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:01.628139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:01.667266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114e1e2-ac2a-4c6a-92d1-e2c386d1af32 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:01.694032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cb33f-28c8-4e3b-8adb-d354c0633867 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:01.706787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28db87-7767-45b7-85a3-c498e2882dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:01.707419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28db87-7767-45b7-85a3-c498e2882dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.707703 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28db87-7767-45b7-85a3-c498e2882dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.707978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28db87-7767-45b7-85a3-c498e2882dc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.709182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28db87-7767-45b7-85a3-c498e2882dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:01.711161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28db87-7767-45b7-85a3-c498e2882dc8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:01.713118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 508/1022] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:01.718422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28db87-7767-45b7-85a3-c498e2882dc8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:01.735707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6a6eb-3543-43df-8630-b3917415a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:01.736071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6a6eb-3543-43df-8630-b3917415a147 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.736348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6a6eb-3543-43df-8630-b3917415a147 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.736616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6a6eb-3543-43df-8630-b3917415a147 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.737332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6a6eb-3543-43df-8630-b3917415a147 None admin] RBAC: Adding query filter params (name=image) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:01.737589 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6a6eb-3543-43df-8630-b3917415a147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:01.738945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6a6eb-3543-43df-8630-b3917415a147 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:01.745135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 515/1023] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:01.750668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6a6eb-3543-43df-8630-b3917415a147 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:01.828039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28db87-7767-45b7-85a3-c498e2882dc8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:01.839241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:01.855693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6a6eb-3543-43df-8630-b3917415a147 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:01.866634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b03d7-4255-4b52-b07f-cf5aa8b5ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:01.867182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:01.868002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b03d7-4255-4b52-b07f-cf5aa8b5ec10 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.868436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b03d7-4255-4b52-b07f-cf5aa8b5ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.868857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b03d7-4255-4b52-b07f-cf5aa8b5ec10 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.869557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b03d7-4255-4b52-b07f-cf5aa8b5ec10 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:01.872395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b03d7-4255-4b52-b07f-cf5aa8b5ec10 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:01.891753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c7093-f260-49c9-ae27-231b67bac6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:01.892360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c7093-f260-49c9-ae27-231b67bac6db None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:01.892654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c7093-f260-49c9-ae27-231b67bac6db None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:01.893060 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c7093-f260-49c9-ae27-231b67bac6db None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:01.893687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c7093-f260-49c9-ae27-231b67bac6db None admin] RBAC: Adding query filter params (type=image) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:01.894039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c7093-f260-49c9-ae27-231b67bac6db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:01.895426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c7093-f260-49c9-ae27-231b67bac6db None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:01.901625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 516/1024] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:01.907210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c7093-f260-49c9-ae27-231b67bac6db None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:01.959788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 509/1025] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:13:01 2022] POST /identity/v3/projects => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:13:02.015839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c7093-f260-49c9-ae27-231b67bac6db None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:02.053848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a31de4-ba75-424b-89d1-0aad52cd5bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:02.054655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a31de4-ba75-424b-89d1-0aad52cd5bdf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:02.055018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a31de4-ba75-424b-89d1-0aad52cd5bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:02.055389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a31de4-ba75-424b-89d1-0aad52cd5bdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:02.056216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a31de4-ba75-424b-89d1-0aad52cd5bdf None admin] RBAC: Adding query filter params (interface=public, service_id=42627b2291f94e27974fa1145e277c55, region_id=RegionOne) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:02.056626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a31de4-ba75-424b-89d1-0aad52cd5bdf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:02.058155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a31de4-ba75-424b-89d1-0aad52cd5bdf None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:02.064305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 517/1026] 10.222.0.84 () {56 vars in 1251 bytes} [Wed Aug 24 11:13:01 2022] GET /identity/v3/endpoints?service_id=42627b2291f94e27974fa1145e277c55&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:02.071059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b03d7-4255-4b52-b07f-cf5aa8b5ec10 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:02.185893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b03d7-4255-4b52-b07f-cf5aa8b5ec10 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:02.221682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed0e64-4540-462f-b8b4-600bf6d546ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:02.222434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed0e64-4540-462f-b8b4-600bf6d546ce None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:02.222829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed0e64-4540-462f-b8b4-600bf6d546ce None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:02.223203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed0e64-4540-462f-b8b4-600bf6d546ce None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:02.223899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed0e64-4540-462f-b8b4-600bf6d546ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:02.225391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed0e64-4540-462f-b8b4-600bf6d546ce None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:02.232253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 510/1027] 10.222.0.84 () {56 vars in 1087 bytes} [Wed Aug 24 11:13:02 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:13:03.436543 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb5e1-1081-45eb-969e-537aa175fcd1 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:03.437140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb5e1-1081-45eb-969e-537aa175fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:03.437140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb5e1-1081-45eb-969e-537aa175fcd1 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:03.438588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 518/1028] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:03.443510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952630bf-2a9a-4764-be18-39aec8898974 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:03.443886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952630bf-2a9a-4764-be18-39aec8898974 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:03.444073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952630bf-2a9a-4764-be18-39aec8898974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:03.462138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-952630bf-2a9a-4764-be18-39aec8898974 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:03.463304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-952630bf-2a9a-4764-be18-39aec8898974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:03.485360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:03.531688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 511/1029] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:03.803020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb5e1-1081-45eb-969e-537aa175fcd1 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:03.844498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:03.906482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb5e1-1081-45eb-969e-537aa175fcd1 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:03.939554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b738e-20cb-412d-a34f-7abbe5cf68c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:03.940178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b738e-20cb-412d-a34f-7abbe5cf68c2 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:03.940449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b738e-20cb-412d-a34f-7abbe5cf68c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:03.940713 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b738e-20cb-412d-a34f-7abbe5cf68c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:03.950800 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b738e-20cb-412d-a34f-7abbe5cf68c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-329b738e-20cb-412d-a34f-7abbe5cf68c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:03.955825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:03.956743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:13:03.957872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:03.961130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 519/1030] 10.222.0.84 () {56 vars in 1120 bytes} [Wed Aug 24 11:13:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:03.963518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952630bf-2a9a-4764-be18-39aec8898974 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.000232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa87a71-74d8-42ae-bc13-39e260314cee None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.000559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa87a71-74d8-42ae-bc13-39e260314cee None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.000789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa87a71-74d8-42ae-bc13-39e260314cee None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.002412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 520/1031] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:13:04.008076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e5209-ec03-4dcc-884c-56476cd94c7e None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.008373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e5209-ec03-4dcc-884c-56476cd94c7e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.008640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e5209-ec03-4dcc-884c-56476cd94c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.027128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d76e5209-ec03-4dcc-884c-56476cd94c7e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:04.028321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d76e5209-ec03-4dcc-884c-56476cd94c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:04.066088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952630bf-2a9a-4764-be18-39aec8898974 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:04.099918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 521/1032] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:13:04.102423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1de199-ee5f-44e0-86f1-206db025dcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:04.103027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1de199-ee5f-44e0-86f1-206db025dcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.103372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1de199-ee5f-44e0-86f1-206db025dcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.103648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1de199-ee5f-44e0-86f1-206db025dcb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.104296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1de199-ee5f-44e0-86f1-206db025dcb2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:04.104593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1de199-ee5f-44e0-86f1-206db025dcb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:04.105982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1de199-ee5f-44e0-86f1-206db025dcb2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:04.118945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 512/1033] 10.222.0.84 () {56 vars in 1115 bytes} [Wed Aug 24 11:13:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:04.120098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e5209-ec03-4dcc-884c-56476cd94c7e None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.148891 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:04.165692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d9a81-92eb-477f-8773-b2c2007a7af4 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.165970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d9a81-92eb-477f-8773-b2c2007a7af4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.166237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d9a81-92eb-477f-8773-b2c2007a7af4 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.168450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 513/1034] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:13:04.172769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fb2df-9576-455e-be81-92836efaa691 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.173077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fb2df-9576-455e-be81-92836efaa691 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.173350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fb2df-9576-455e-be81-92836efaa691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.191754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-eb3fb2df-9576-455e-be81-92836efaa691 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:04.193113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-eb3fb2df-9576-455e-be81-92836efaa691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:04.206333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:04.229151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e5209-ec03-4dcc-884c-56476cd94c7e None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:04.265293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 514/1035] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:13:04.266863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c5c02-8797-4765-9608-636af6f85312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:04.267716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c5c02-8797-4765-9608-636af6f85312 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.268191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c5c02-8797-4765-9608-636af6f85312 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.268636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c5c02-8797-4765-9608-636af6f85312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.279795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c5c02-8797-4765-9608-636af6f85312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:04.281658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c5c02-8797-4765-9608-636af6f85312 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-849c5c02-8797-4765-9608-636af6f85312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:13:04.296401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:04.297496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:13:04.298601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:13:04.298601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:04.298601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:04.300239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 522/1036] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:04.305883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fb2df-9576-455e-be81-92836efaa691 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.368854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c5c02-8797-4765-9608-636af6f85312 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.410919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fb2df-9576-455e-be81-92836efaa691 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:04.446838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa63cd-da32-45e8-8d45-47d565dc2578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:04.447503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa63cd-da32-45e8-8d45-47d565dc2578 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.447734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa63cd-da32-45e8-8d45-47d565dc2578 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.448017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa63cd-da32-45e8-8d45-47d565dc2578 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.448651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa63cd-da32-45e8-8d45-47d565dc2578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:04.449024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa63cd-da32-45e8-8d45-47d565dc2578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:04.450375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa63cd-da32-45e8-8d45-47d565dc2578 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:04.455571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:04.463155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 515/1037] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:04.469851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa63cd-da32-45e8-8d45-47d565dc2578 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.476083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c5c02-8797-4765-9608-636af6f85312 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:04.481026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:04.513089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462a6c5-9371-4443-abb1-0454be084271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:04.513999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462a6c5-9371-4443-abb1-0454be084271 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.514438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462a6c5-9371-4443-abb1-0454be084271 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.514862 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462a6c5-9371-4443-abb1-0454be084271 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.516117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462a6c5-9371-4443-abb1-0454be084271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:04.517750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462a6c5-9371-4443-abb1-0454be084271 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:04.519617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 523/1038] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:04.525556 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462a6c5-9371-4443-abb1-0454be084271 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.572483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa63cd-da32-45e8-8d45-47d565dc2578 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:04.610747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120d3b0-372a-4536-8d81-c751b12067d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:04.611409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120d3b0-372a-4536-8d81-c751b12067d0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.611695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120d3b0-372a-4536-8d81-c751b12067d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.611971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120d3b0-372a-4536-8d81-c751b12067d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.620819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120d3b0-372a-4536-8d81-c751b12067d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:04.622238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120d3b0-372a-4536-8d81-c751b12067d0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3120d3b0-372a-4536-8d81-c751b12067d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:13:04.633220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:04.634193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:13:04.640220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:04.640942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462a6c5-9371-4443-abb1-0454be084271 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:04.641046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 516/1039] 10.222.0.84 () {56 vars in 1093 bytes} [Wed Aug 24 11:13:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:13:04.641046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120d3b0-372a-4536-8d81-c751b12067d0 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.676141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f94cd-b546-44c5-b5df-7ee84ef6fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:04.676747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f94cd-b546-44c5-b5df-7ee84ef6fe01 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.677057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f94cd-b546-44c5-b5df-7ee84ef6fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.677323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f94cd-b546-44c5-b5df-7ee84ef6fe01 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.677966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f94cd-b546-44c5-b5df-7ee84ef6fe01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:04.679600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f94cd-b546-44c5-b5df-7ee84ef6fe01 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:04.701479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:04.734834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120d3b0-372a-4536-8d81-c751b12067d0 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:04.766402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ec557-9979-46e5-9aae-fc94fefdb5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:04.767016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ec557-9979-46e5-9aae-fc94fefdb5ed None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.767285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ec557-9979-46e5-9aae-fc94fefdb5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.767550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ec557-9979-46e5-9aae-fc94fefdb5ed None admin] PATH_INFO: `/v3/services/image` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.768116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ec557-9979-46e5-9aae-fc94fefdb5ed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:04.769489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ec557-9979-46e5-9aae-fc94fefdb5ed None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-405ec557-9979-46e5-9aae-fc94fefdb5ed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:04.778328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:04.783269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:04.784093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 517/1040] 10.222.0.84 () {56 vars in 1105 bytes} [Wed Aug 24 11:13:04 2022] GET /identity/v3/services/image => generated 86 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:04.785246 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ec557-9979-46e5-9aae-fc94fefdb5ed None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.797023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 524/1041] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:13:04 2022] POST /identity/v3/users => generated 285 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:13:04.797023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f94cd-b546-44c5-b5df-7ee84ef6fe01 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.798755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:04.905029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ec557-9979-46e5-9aae-fc94fefdb5ed None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:04.919356 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f94cd-b546-44c5-b5df-7ee84ef6fe01 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:04.951040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:04.956677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8493ca5-6700-4eab-b807-044dea3ec596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:04.957314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8493ca5-6700-4eab-b807-044dea3ec596 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.957592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8493ca5-6700-4eab-b807-044dea3ec596 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.964007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8493ca5-6700-4eab-b807-044dea3ec596 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.964007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8493ca5-6700-4eab-b807-044dea3ec596 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:04.964007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8493ca5-6700-4eab-b807-044dea3ec596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:04.964007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8493ca5-6700-4eab-b807-044dea3ec596 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:04.964505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800f932-7400-4a48-a46b-b7a5e382d166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:04.964505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800f932-7400-4a48-a46b-b7a5e382d166 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:04.964505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800f932-7400-4a48-a46b-b7a5e382d166 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:04.964505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800f932-7400-4a48-a46b-b7a5e382d166 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:04.965017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800f932-7400-4a48-a46b-b7a5e382d166 None admin] RBAC: Adding query filter params (name=image) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:04.965017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800f932-7400-4a48-a46b-b7a5e382d166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:04.965017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800f932-7400-4a48-a46b-b7a5e382d166 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:04.969394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 525/1042] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:04.974711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800f932-7400-4a48-a46b-b7a5e382d166 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:04.983934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 518/1043] 10.222.0.84 () {56 vars in 1133 bytes} [Wed Aug 24 11:13:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:04.990326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8493ca5-6700-4eab-b807-044dea3ec596 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:05.055709 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:05.089936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800f932-7400-4a48-a46b-b7a5e382d166 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:05.104603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8493ca5-6700-4eab-b807-044dea3ec596 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:05.128253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abdba8-1a85-4a44-9e5e-51262da5e64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:05.128895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abdba8-1a85-4a44-9e5e-51262da5e64d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:05.129169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abdba8-1a85-4a44-9e5e-51262da5e64d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:05.129433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abdba8-1a85-4a44-9e5e-51262da5e64d None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:05.130199 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abdba8-1a85-4a44-9e5e-51262da5e64d None admin] RBAC: Adding query filter params (type=image) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:05.130781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abdba8-1a85-4a44-9e5e-51262da5e64d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:05.132091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abdba8-1a85-4a44-9e5e-51262da5e64d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:05.137956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 526/1044] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:05.143483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abdba8-1a85-4a44-9e5e-51262da5e64d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:05.144505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5ab3c-9608-4211-b1e9-2711be96d581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:05.145137 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5ab3c-9608-4211-b1e9-2711be96d581 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:05.145418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5ab3c-9608-4211-b1e9-2711be96d581 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:05.145691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5ab3c-9608-4211-b1e9-2711be96d581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:05.206157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:05.216886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5ab3c-9608-4211-b1e9-2711be96d581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:05.221923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c5ab3c-9608-4211-b1e9-2711be96d581 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:05.253959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abdba8-1a85-4a44-9e5e-51262da5e64d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:05.291388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:05.298394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27093b0-7998-4893-acf4-55374e33995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:05.299006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27093b0-7998-4893-acf4-55374e33995b None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:05.299286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27093b0-7998-4893-acf4-55374e33995b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:05.299555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27093b0-7998-4893-acf4-55374e33995b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:05.300212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27093b0-7998-4893-acf4-55374e33995b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:05.301615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27093b0-7998-4893-acf4-55374e33995b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:05.340045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 519/1045] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:13:05.345492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5ab3c-9608-4211-b1e9-2711be96d581 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:05.387285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 527/1046] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:13:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:13:05.445719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:05.456035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c5ab3c-9608-4211-b1e9-2711be96d581 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:05.503961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6006f2-b56d-42e2-9463-06d253a71971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:05.504512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6006f2-b56d-42e2-9463-06d253a71971 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:05.504832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6006f2-b56d-42e2-9463-06d253a71971 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:05.505088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6006f2-b56d-42e2-9463-06d253a71971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:05.516852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6006f2-b56d-42e2-9463-06d253a71971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:05.518410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6006f2-b56d-42e2-9463-06d253a71971 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-3d6006f2-b56d-42e2-9463-06d253a71971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:13:05.535110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:05.536088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:13:05.537642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:13:05.537642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:05.537642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:05.537642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 520/1047] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:13:05.542599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27093b0-7998-4893-acf4-55374e33995b None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:05.648394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27093b0-7998-4893-acf4-55374e33995b None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:05.684375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce2de3-c73e-4038-bdde-984f012a54ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:05.685005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce2de3-c73e-4038-bdde-984f012a54ff None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:05.685283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce2de3-c73e-4038-bdde-984f012a54ff None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:05.685553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce2de3-c73e-4038-bdde-984f012a54ff None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:05.686185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce2de3-c73e-4038-bdde-984f012a54ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:05.686483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce2de3-c73e-4038-bdde-984f012a54ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:05.687854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce2de3-c73e-4038-bdde-984f012a54ff None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:05.699892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 528/1048] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:05.705762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6006f2-b56d-42e2-9463-06d253a71971 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:05.810340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6006f2-b56d-42e2-9463-06d253a71971 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:05.844469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398850-4c05-4c8b-a54c-73c457f5eb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:05.845130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398850-4c05-4c8b-a54c-73c457f5eb59 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:05.845405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398850-4c05-4c8b-a54c-73c457f5eb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:05.845673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398850-4c05-4c8b-a54c-73c457f5eb59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:05.852500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd398850-4c05-4c8b-a54c-73c457f5eb59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:05.854122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd398850-4c05-4c8b-a54c-73c457f5eb59 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:05.855873 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-fd398850-4c05-4c8b-a54c-73c457f5eb59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:13:05.866757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:05.872666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:05.875801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:13:05.875801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:13:05.875801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:13:05.875801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:13:05.875801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:13:05.875801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:13:05.875801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:13:05.875801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:05.875801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 521/1049] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:13:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:05.876300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce2de3-c73e-4038-bdde-984f012a54ff None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:05.927219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:05.981760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce2de3-c73e-4038-bdde-984f012a54ff None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:06.017354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff9595-5008-4205-bfb0-cb1bae2da891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:06.017949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff9595-5008-4205-bfb0-cb1bae2da891 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:06.018223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff9595-5008-4205-bfb0-cb1bae2da891 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:06.018484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff9595-5008-4205-bfb0-cb1bae2da891 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:06.019158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff9595-5008-4205-bfb0-cb1bae2da891 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:06.019454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff9595-5008-4205-bfb0-cb1bae2da891 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:06.020901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff9595-5008-4205-bfb0-cb1bae2da891 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:06.038160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 529/1050] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:13:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:06.039432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398850-4c05-4c8b-a54c-73c457f5eb59 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:06.140677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398850-4c05-4c8b-a54c-73c457f5eb59 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:06.173064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52d526-5611-4b3d-8bb4-3e122fddab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:06.173652 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52d526-5611-4b3d-8bb4-3e122fddab0f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:06.173928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52d526-5611-4b3d-8bb4-3e122fddab0f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:06.174193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52d526-5611-4b3d-8bb4-3e122fddab0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:06.174899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52d526-5611-4b3d-8bb4-3e122fddab0f None admin] RBAC: Adding query filter params (role.id=b2da7bb417d94f45a2a4e8a2befa0988, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=f58a559951c94d68951da78c82341a00) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:06.175200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52d526-5611-4b3d-8bb4-3e122fddab0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:06.176591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52d526-5611-4b3d-8bb4-3e122fddab0f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:06.183661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 522/1051] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:13:06 2022] GET /identity/v3/role_assignments?user.id=f58a559951c94d68951da78c82341a00&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=b2da7bb417d94f45a2a4e8a2befa0988 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:06.808004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e89abe-93be-496f-9c44-0cc277855005 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:06.808515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e89abe-93be-496f-9c44-0cc277855005 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:06.808515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e89abe-93be-496f-9c44-0cc277855005 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:06.809962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 530/1052] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:06.815128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f8d37-b6e4-42fd-abee-19629770514a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:06.815509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f8d37-b6e4-42fd-abee-19629770514a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:06.815685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f8d37-b6e4-42fd-abee-19629770514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:06.833489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1f5f8d37-b6e4-42fd-abee-19629770514a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:06.834670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1f5f8d37-b6e4-42fd-abee-19629770514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:06.891695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:06.907185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 523/1053] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:07.176513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e89abe-93be-496f-9c44-0cc277855005 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:07.288669 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e89abe-93be-496f-9c44-0cc277855005 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:07.297880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:07.323913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94334d2-afe1-4fd0-8f7f-1042a3930113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:07.324674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94334d2-afe1-4fd0-8f7f-1042a3930113 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:07.324980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94334d2-afe1-4fd0-8f7f-1042a3930113 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:07.325245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94334d2-afe1-4fd0-8f7f-1042a3930113 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:07.335659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d2a4c-6735-47a5-9c21-91fd9aff8b3e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:07.335966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d2a4c-6735-47a5-9c21-91fd9aff8b3e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:07.336200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d2a4c-6735-47a5-9c21-91fd9aff8b3e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:07.337706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 524/1054] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:07.337816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94334d2-afe1-4fd0-8f7f-1042a3930113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-b94334d2-afe1-4fd0-8f7f-1042a3930113 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:07.342392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:07.343317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:07.345827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:07.346484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c977874-4ac7-4f35-b1f6-a8d1da87d5ca None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:07.346484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c977874-4ac7-4f35-b1f6-a8d1da87d5ca None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:07.346484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c977874-4ac7-4f35-b1f6-a8d1da87d5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:07.346661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 531/1055] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:13:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:07.349349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94334d2-afe1-4fd0-8f7f-1042a3930113 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:07.364711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1c977874-4ac7-4f35-b1f6-a8d1da87d5ca None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:07.365913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1c977874-4ac7-4f35-b1f6-a8d1da87d5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:07.453386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 525/1056] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:13:07.460608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94334d2-afe1-4fd0-8f7f-1042a3930113 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:07.498025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96434b-bfc4-410a-8985-c0786905bd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:07.498756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96434b-bfc4-410a-8985-c0786905bd86 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:07.498869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96434b-bfc4-410a-8985-c0786905bd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:07.499144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96434b-bfc4-410a-8985-c0786905bd86 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:07.499801 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96434b-bfc4-410a-8985-c0786905bd86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:07.500100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96434b-bfc4-410a-8985-c0786905bd86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:07.501498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96434b-bfc4-410a-8985-c0786905bd86 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:07.510971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 532/1057] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:13:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:07.516576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c977874-4ac7-4f35-b1f6-a8d1da87d5ca None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:07.619529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c977874-4ac7-4f35-b1f6-a8d1da87d5ca None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:07.642838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:07.659659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee8fdf-dd43-485d-b635-994634c8d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:07.660299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee8fdf-dd43-485d-b635-994634c8d7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:07.660581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee8fdf-dd43-485d-b635-994634c8d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:07.660885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee8fdf-dd43-485d-b635-994634c8d7e2 None admin] PATH_INFO: `/v3/users/28cd3a5305e442a690322c76d5fbfbc1` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:07.676138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee8fdf-dd43-485d-b635-994634c8d7e2 None admin] RBAC: Authorizing `identity:get_user(user_id=28cd3a5305e442a690322c76d5fbfbc1)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:07.677605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee8fdf-dd43-485d-b635-994634c8d7e2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:07.684626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 526/1058] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:07 2022] GET /identity/v3/users/28cd3a5305e442a690322c76d5fbfbc1 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:07.685684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96434b-bfc4-410a-8985-c0786905bd86 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:07.718594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:07.731300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee8fdf-dd43-485d-b635-994634c8d7e2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:07.797977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96434b-bfc4-410a-8985-c0786905bd86 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:07.833743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67060f-bbaa-4e85-8cc2-787c61edcd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:07.834366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67060f-bbaa-4e85-8cc2-787c61edcd25 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:07.834608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67060f-bbaa-4e85-8cc2-787c61edcd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:07.834880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67060f-bbaa-4e85-8cc2-787c61edcd25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:07.844286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee8fdf-dd43-485d-b635-994634c8d7e2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:07.880568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d166d-8809-4df4-b75f-92b7ddcb9945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:07.881724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d166d-8809-4df4-b75f-92b7ddcb9945 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:07.882283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d166d-8809-4df4-b75f-92b7ddcb9945 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:07.882790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d166d-8809-4df4-b75f-92b7ddcb9945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:07.906663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67060f-bbaa-4e85-8cc2-787c61edcd25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:07.908119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67060f-bbaa-4e85-8cc2-787c61edcd25 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:07.935360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:07.937152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:07.956366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d166d-8809-4df4-b75f-92b7ddcb9945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:07.958109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d166d-8809-4df4-b75f-92b7ddcb9945 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:08.013021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 533/1059] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:13:08.018919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67060f-bbaa-4e85-8cc2-787c61edcd25 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:08.068622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 527/1060] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 11:13:08.069886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d166d-8809-4df4-b75f-92b7ddcb9945 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:08.119939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67060f-bbaa-4e85-8cc2-787c61edcd25 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:08.149277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:08.151556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:08.154541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a37d28-3122-46aa-9fea-9bfa5ea92f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:08.155157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a37d28-3122-46aa-9fea-9bfa5ea92f3b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:08.155430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a37d28-3122-46aa-9fea-9bfa5ea92f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:08.155693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a37d28-3122-46aa-9fea-9bfa5ea92f3b None admin] PATH_INFO: `/v3/projects/286f0726633b449d936b1774cc84e8dc` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:08.156781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a37d28-3122-46aa-9fea-9bfa5ea92f3b None admin] RBAC: Authorizing `identity:get_project(project_id=286f0726633b449d936b1774cc84e8dc)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:08.158219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a37d28-3122-46aa-9fea-9bfa5ea92f3b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:08.159858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 534/1061] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:08 2022] GET /identity/v3/projects/286f0726633b449d936b1774cc84e8dc => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:08.161091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9a3e-c839-43c8-85ca-821c9ddbbf6d None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:08.161329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9a3e-c839-43c8-85ca-821c9ddbbf6d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:08.161533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9a3e-c839-43c8-85ca-821c9ddbbf6d None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:08.162765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 535/1062] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:08.169744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9a3e-c839-43c8-85ca-821c9ddbbf6d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:08.171895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d166d-8809-4df4-b75f-92b7ddcb9945 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:08.205918 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1a459-9890-4816-8242-aa678b5b8908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:08.206542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1a459-9890-4816-8242-aa678b5b8908 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:08.206818 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1a459-9890-4816-8242-aa678b5b8908 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:08.207088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1a459-9890-4816-8242-aa678b5b8908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:08.217976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1a459-9890-4816-8242-aa678b5b8908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:08.219415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1a459-9890-4816-8242-aa678b5b8908 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-a1d1a459-9890-4816-8242-aa678b5b8908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:13:08.234046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:08.235006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:13:08.235896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:13:08.235896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:08.235896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:08.235896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 528/1063] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:08.236408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f886e8-ed9c-4b15-9473-b3ae15fab00d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:08.236688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f886e8-ed9c-4b15-9473-b3ae15fab00d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:08.236981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f886e8-ed9c-4b15-9473-b3ae15fab00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:08.254856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-62f886e8-ed9c-4b15-9473-b3ae15fab00d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:08.256042 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-62f886e8-ed9c-4b15-9473-b3ae15fab00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:08.274735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9a3e-c839-43c8-85ca-821c9ddbbf6d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:08.308252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e94f7-faca-4297-bf23-cbcc43d19042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:08.308970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e94f7-faca-4297-bf23-cbcc43d19042 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:08.309243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e94f7-faca-4297-bf23-cbcc43d19042 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:08.309504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e94f7-faca-4297-bf23-cbcc43d19042 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:08.310211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e94f7-faca-4297-bf23-cbcc43d19042 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=286f0726633b449d936b1774cc84e8dc, user.id=28cd3a5305e442a690322c76d5fbfbc1) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:08.310508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e94f7-faca-4297-bf23-cbcc43d19042 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:08.311908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e94f7-faca-4297-bf23-cbcc43d19042 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:08.317103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 536/1064] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:13:08 2022] GET /identity/v3/role_assignments?user.id=28cd3a5305e442a690322c76d5fbfbc1&scope.project.id=286f0726633b449d936b1774cc84e8dc&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:08.317864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e94f7-faca-4297-bf23-cbcc43d19042 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:08.329755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 529/1065] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:08.390114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:08.430104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e94f7-faca-4297-bf23-cbcc43d19042 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:08.467974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185900-e12d-446e-a66e-5c2e2a6a69b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:08.468578 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185900-e12d-446e-a66e-5c2e2a6a69b1 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:08.468874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185900-e12d-446e-a66e-5c2e2a6a69b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:08.469141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185900-e12d-446e-a66e-5c2e2a6a69b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:08.469781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53185900-e12d-446e-a66e-5c2e2a6a69b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:08.470074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53185900-e12d-446e-a66e-5c2e2a6a69b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:08.471422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53185900-e12d-446e-a66e-5c2e2a6a69b1 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:08.483696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 537/1066] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:08.606378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f886e8-ed9c-4b15-9473-b3ae15fab00d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:08.679625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:08.720107 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f886e8-ed9c-4b15-9473-b3ae15fab00d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:08.755885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea51223-9c6b-47ac-b0db-80270fe66f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:08.756510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea51223-9c6b-47ac-b0db-80270fe66f45 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:08.757375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea51223-9c6b-47ac-b0db-80270fe66f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:08.757642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea51223-9c6b-47ac-b0db-80270fe66f45 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:08.768137 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea51223-9c6b-47ac-b0db-80270fe66f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-eea51223-9c6b-47ac-b0db-80270fe66f45 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:08.773660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:08.774649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:08.775487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 530/1067] 10.222.0.84 () {56 vars in 1120 bytes} [Wed Aug 24 11:13:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:08.782360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185900-e12d-446e-a66e-5c2e2a6a69b1 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:08.879807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185900-e12d-446e-a66e-5c2e2a6a69b1 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:08.901943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:08.912741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecd41f-8863-4bc1-bb37-8067ff8dc39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:08.913410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecd41f-8863-4bc1-bb37-8067ff8dc39e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:08.913853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecd41f-8863-4bc1-bb37-8067ff8dc39e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:08.914120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecd41f-8863-4bc1-bb37-8067ff8dc39e None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:08.914767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecd41f-8863-4bc1-bb37-8067ff8dc39e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:08.915063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecd41f-8863-4bc1-bb37-8067ff8dc39e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:08.916461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecd41f-8863-4bc1-bb37-8067ff8dc39e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:08.925617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 538/1068] 10.222.0.84 () {56 vars in 1115 bytes} [Wed Aug 24 11:13:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:08.931351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea51223-9c6b-47ac-b0db-80270fe66f45 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:09.038534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea51223-9c6b-47ac-b0db-80270fe66f45 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:09.060303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:09.071975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975caeb-7403-4f47-8f27-3e2ee86a72d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:09.072581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975caeb-7403-4f47-8f27-3e2ee86a72d7 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:09.072884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975caeb-7403-4f47-8f27-3e2ee86a72d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:09.073152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975caeb-7403-4f47-8f27-3e2ee86a72d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:09.082694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975caeb-7403-4f47-8f27-3e2ee86a72d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:09.084111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975caeb-7403-4f47-8f27-3e2ee86a72d7 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-f975caeb-7403-4f47-8f27-3e2ee86a72d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:13:09.099067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:09.100029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:13:09.101151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:13:09.101151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:09.101151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:09.105089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 531/1069] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:09.106126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecd41f-8863-4bc1-bb37-8067ff8dc39e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:09.203161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecd41f-8863-4bc1-bb37-8067ff8dc39e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:09.236266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000feac6-50e3-4ddd-92be-f066e9252b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:09.236950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000feac6-50e3-4ddd-92be-f066e9252b27 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:09.237226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000feac6-50e3-4ddd-92be-f066e9252b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:09.237489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000feac6-50e3-4ddd-92be-f066e9252b27 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:09.238112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000feac6-50e3-4ddd-92be-f066e9252b27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:09.238403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000feac6-50e3-4ddd-92be-f066e9252b27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:09.239753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000feac6-50e3-4ddd-92be-f066e9252b27 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:09.251261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 539/1070] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:09.257683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975caeb-7403-4f47-8f27-3e2ee86a72d7 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:09.359343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975caeb-7403-4f47-8f27-3e2ee86a72d7 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:09.395135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be4a0b-9295-4b4f-8eb2-9fa428d0fe3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:09.395796 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be4a0b-9295-4b4f-8eb2-9fa428d0fe3f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:09.396079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be4a0b-9295-4b4f-8eb2-9fa428d0fe3f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:09.396353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be4a0b-9295-4b4f-8eb2-9fa428d0fe3f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:09.406302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be4a0b-9295-4b4f-8eb2-9fa428d0fe3f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:09.407757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be4a0b-9295-4b4f-8eb2-9fa428d0fe3f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-93be4a0b-9295-4b4f-8eb2-9fa428d0fe3f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:13:09.420182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:09.421231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:09.422193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 532/1071] 10.222.0.84 () {56 vars in 1093 bytes} [Wed Aug 24 11:13:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:13:09.427180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000feac6-50e3-4ddd-92be-f066e9252b27 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:09.472422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:09.525032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000feac6-50e3-4ddd-92be-f066e9252b27 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:09.558628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301aaee-f12d-4bbc-a38e-15f1b9a69ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:09.559226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301aaee-f12d-4bbc-a38e-15f1b9a69ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:09.559502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301aaee-f12d-4bbc-a38e-15f1b9a69ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:09.559769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301aaee-f12d-4bbc-a38e-15f1b9a69ce2 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:09.560482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301aaee-f12d-4bbc-a38e-15f1b9a69ce2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:09.560782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301aaee-f12d-4bbc-a38e-15f1b9a69ce2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:09.562215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301aaee-f12d-4bbc-a38e-15f1b9a69ce2 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:09.578749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 540/1072] 10.222.0.84 () {56 vars in 1133 bytes} [Wed Aug 24 11:13:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:09.584557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be4a0b-9295-4b4f-8eb2-9fa428d0fe3f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:09.619923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:09.685404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be4a0b-9295-4b4f-8eb2-9fa428d0fe3f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:09.720959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34fdc7-1083-4a71-a0e6-2ec3cfb1b973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:09.721581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34fdc7-1083-4a71-a0e6-2ec3cfb1b973 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:09.721858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34fdc7-1083-4a71-a0e6-2ec3cfb1b973 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:09.722146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34fdc7-1083-4a71-a0e6-2ec3cfb1b973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:09.733139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34fdc7-1083-4a71-a0e6-2ec3cfb1b973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:09.734750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34fdc7-1083-4a71-a0e6-2ec3cfb1b973 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-7e34fdc7-1083-4a71-a0e6-2ec3cfb1b973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:13:09.749447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:09.750545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:13:09.751468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:13:09.751468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:09.751468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:09.751468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 533/1073] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:09.756427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301aaee-f12d-4bbc-a38e-15f1b9a69ce2 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:09.852424 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301aaee-f12d-4bbc-a38e-15f1b9a69ce2 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:09.853979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:09.887680 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e27e1a-9d64-4fb9-9fce-fd0acbaadb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:09.888309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e27e1a-9d64-4fb9-9fce-fd0acbaadb5e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:09.888530 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e27e1a-9d64-4fb9-9fce-fd0acbaadb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:09.888804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e27e1a-9d64-4fb9-9fce-fd0acbaadb5e None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:09.889442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e27e1a-9d64-4fb9-9fce-fd0acbaadb5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:09.889736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e27e1a-9d64-4fb9-9fce-fd0acbaadb5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:09.891078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e27e1a-9d64-4fb9-9fce-fd0acbaadb5e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:09.902594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 541/1074] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:09.908281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34fdc7-1083-4a71-a0e6-2ec3cfb1b973 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:09.993693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:10.007295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34fdc7-1083-4a71-a0e6-2ec3cfb1b973 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:10.042304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea929965-3d15-47e8-89d7-9cffe5ed2a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:10.042929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea929965-3d15-47e8-89d7-9cffe5ed2a42 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:10.043204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea929965-3d15-47e8-89d7-9cffe5ed2a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:10.043476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea929965-3d15-47e8-89d7-9cffe5ed2a42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:10.050089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea929965-3d15-47e8-89d7-9cffe5ed2a42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:10.051513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea929965-3d15-47e8-89d7-9cffe5ed2a42 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-ea929965-3d15-47e8-89d7-9cffe5ed2a42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:13:10.062685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:10.063547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:10.064562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:13:10.064562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:13:10.064562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:13:10.064562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:13:10.064562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:13:10.064562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:13:10.064562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:13:10.064562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:10.064562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 534/1075] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:10.069556 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e27e1a-9d64-4fb9-9fce-fd0acbaadb5e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:10.174227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e27e1a-9d64-4fb9-9fce-fd0acbaadb5e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:10.206461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcda02-e433-4d96-ac51-c224e957d01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:10.207187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcda02-e433-4d96-ac51-c224e957d01b None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:10.207464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcda02-e433-4d96-ac51-c224e957d01b None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:10.207731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcda02-e433-4d96-ac51-c224e957d01b None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:10.208413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcda02-e433-4d96-ac51-c224e957d01b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:10.208710 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcda02-e433-4d96-ac51-c224e957d01b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:10.210656 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcda02-e433-4d96-ac51-c224e957d01b None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:10.222507 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 542/1076] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:13:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:10.229323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea929965-3d15-47e8-89d7-9cffe5ed2a42 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:10.281512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e66688-5ed1-411b-a66b-14805c666e9a None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:10.281717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e66688-5ed1-411b-a66b-14805c666e9a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:10.281983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e66688-5ed1-411b-a66b-14805c666e9a None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:10.283295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 543/1077] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:13:10.288115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f371d1-e2c8-42e7-a535-028a95b6fcf8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:10.288396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f371d1-e2c8-42e7-a535-028a95b6fcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:10.288671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f371d1-e2c8-42e7-a535-028a95b6fcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:10.305976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-23f371d1-e2c8-42e7-a535-028a95b6fcf8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:10.307161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-23f371d1-e2c8-42e7-a535-028a95b6fcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:10.323689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:10.331323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea929965-3d15-47e8-89d7-9cffe5ed2a42 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:10.365109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d00de6-d527-4df7-adaf-38682ce40e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:10.365753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d00de6-d527-4df7-adaf-38682ce40e20 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:10.366211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d00de6-d527-4df7-adaf-38682ce40e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:10.366496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d00de6-d527-4df7-adaf-38682ce40e20 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/f58a559951c94d68951da78c82341a00/roles/b2da7bb417d94f45a2a4e8a2befa0988` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:10.368596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d00de6-d527-4df7-adaf-38682ce40e20 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=f58a559951c94d68951da78c82341a00, role_id=b2da7bb417d94f45a2a4e8a2befa0988)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:10.370215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d00de6-d527-4df7-adaf-38682ce40e20 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:10.374509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 544/1078] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:13:10.426617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 535/1079] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:13:10 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/f58a559951c94d68951da78c82341a00/roles/b2da7bb417d94f45a2a4e8a2befa0988 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:13:10.646629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f371d1-e2c8-42e7-a535-028a95b6fcf8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:10.745088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f371d1-e2c8-42e7-a535-028a95b6fcf8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:10.778504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e605a2-a34e-4a82-9548-b08a7a604c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:10.779122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e605a2-a34e-4a82-9548-b08a7a604c8e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:10.779399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e605a2-a34e-4a82-9548-b08a7a604c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:10.779667 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e605a2-a34e-4a82-9548-b08a7a604c8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:10.789746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e605a2-a34e-4a82-9548-b08a7a604c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-b3e605a2-a34e-4a82-9548-b08a7a604c8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:10.794398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:10.795479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:10.796504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:10.800482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 545/1080] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:13:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:10.801505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d00de6-d527-4df7-adaf-38682ce40e20 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:10.811350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:10.902461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d00de6-d527-4df7-adaf-38682ce40e20 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:10.935560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eec3e0-b188-4502-a39b-29f86fd06c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:10.936158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eec3e0-b188-4502-a39b-29f86fd06c62 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:10.936440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eec3e0-b188-4502-a39b-29f86fd06c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:10.936711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eec3e0-b188-4502-a39b-29f86fd06c62 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:10.937394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eec3e0-b188-4502-a39b-29f86fd06c62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:10.937696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eec3e0-b188-4502-a39b-29f86fd06c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:10.939050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eec3e0-b188-4502-a39b-29f86fd06c62 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:10.948208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 536/1081] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:13:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:10.953791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e605a2-a34e-4a82-9548-b08a7a604c8e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:10.969020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:11.056060 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e605a2-a34e-4a82-9548-b08a7a604c8e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:11.089484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86741511-b591-4a61-b25d-4063527c2f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:11.090091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86741511-b591-4a61-b25d-4063527c2f35 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:11.090363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86741511-b591-4a61-b25d-4063527c2f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:11.090625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86741511-b591-4a61-b25d-4063527c2f35 None admin] PATH_INFO: `/v3/users/28cd3a5305e442a690322c76d5fbfbc1` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:11.091715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86741511-b591-4a61-b25d-4063527c2f35 None admin] RBAC: Authorizing `identity:get_user(user_id=28cd3a5305e442a690322c76d5fbfbc1)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:11.093155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86741511-b591-4a61-b25d-4063527c2f35 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:11.095030 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 546/1082] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:10 2022] GET /identity/v3/users/28cd3a5305e442a690322c76d5fbfbc1 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:11.101108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eec3e0-b188-4502-a39b-29f86fd06c62 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:11.165738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:11.198998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eec3e0-b188-4502-a39b-29f86fd06c62 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:11.231784 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc014d3-d3ab-4427-a216-ed88de8b03e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:11.232394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc014d3-d3ab-4427-a216-ed88de8b03e5 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:11.232664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc014d3-d3ab-4427-a216-ed88de8b03e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:11.232960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc014d3-d3ab-4427-a216-ed88de8b03e5 None admin] PATH_INFO: `/v3/projects/286f0726633b449d936b1774cc84e8dc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:11.234304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc014d3-d3ab-4427-a216-ed88de8b03e5 None admin] RBAC: Authorizing `identity:get_project(project_id=286f0726633b449d936b1774cc84e8dc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:11.235710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc014d3-d3ab-4427-a216-ed88de8b03e5 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:11.237339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 537/1083] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:11 2022] GET /identity/v3/projects/286f0726633b449d936b1774cc84e8dc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:11.244460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86741511-b591-4a61-b25d-4063527c2f35 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:11.322940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:11.340726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86741511-b591-4a61-b25d-4063527c2f35 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:11.375218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8373f2-ccea-4fdd-b300-4a52bfb555f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:11.375868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8373f2-ccea-4fdd-b300-4a52bfb555f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:11.376141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8373f2-ccea-4fdd-b300-4a52bfb555f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:11.376404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8373f2-ccea-4fdd-b300-4a52bfb555f3 None admin] PATH_INFO: `/v3/projects/286f0726633b449d936b1774cc84e8dc/users/28cd3a5305e442a690322c76d5fbfbc1/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:11.378315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8373f2-ccea-4fdd-b300-4a52bfb555f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=286f0726633b449d936b1774cc84e8dc, user_id=28cd3a5305e442a690322c76d5fbfbc1, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:11.379738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8373f2-ccea-4fdd-b300-4a52bfb555f3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:11.453840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 547/1084] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:13:11 2022] PUT /identity/v3/projects/286f0726633b449d936b1774cc84e8dc/users/28cd3a5305e442a690322c76d5fbfbc1/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:13:12.342518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29999-a066-4b8a-82bb-9e2f8d17979e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:12.343056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29999-a066-4b8a-82bb-9e2f8d17979e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:12.343056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29999-a066-4b8a-82bb-9e2f8d17979e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:12.344373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 538/1085] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:13:12.349518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95124f-b17d-4963-981f-4d8ed150fbba None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:12.349897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95124f-b17d-4963-981f-4d8ed150fbba None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:12.350170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95124f-b17d-4963-981f-4d8ed150fbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:12.369001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ff95124f-b17d-4963-981f-4d8ed150fbba None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:12.370173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ff95124f-b17d-4963-981f-4d8ed150fbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:12.434080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 548/1086] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:13:12.701125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29999-a066-4b8a-82bb-9e2f8d17979e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:12.801173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29999-a066-4b8a-82bb-9e2f8d17979e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:12.834225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca142a-f01b-4e90-b971-8314653c876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:12.834985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca142a-f01b-4e90-b971-8314653c876e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:12.835343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca142a-f01b-4e90-b971-8314653c876e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:12.835614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca142a-f01b-4e90-b971-8314653c876e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:12.838323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:12.846500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca142a-f01b-4e90-b971-8314653c876e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-95ca142a-f01b-4e90-b971-8314653c876e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:12.851870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:12.854380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:12.855218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 539/1087] 10.222.0.84 () {56 vars in 1120 bytes} [Wed Aug 24 11:13:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:12.859115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95124f-b17d-4963-981f-4d8ed150fbba None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:12.926366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:12.955157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95124f-b17d-4963-981f-4d8ed150fbba None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:12.989245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c3f46-15a6-4299-8886-0f8272ca38d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:12.989841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c3f46-15a6-4299-8886-0f8272ca38d3 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:12.990190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c3f46-15a6-4299-8886-0f8272ca38d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:12.990474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c3f46-15a6-4299-8886-0f8272ca38d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:12.991154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c3f46-15a6-4299-8886-0f8272ca38d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:12.991557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c3f46-15a6-4299-8886-0f8272ca38d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:12.992924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c3f46-15a6-4299-8886-0f8272ca38d3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:13.001339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 549/1088] 10.222.0.84 () {56 vars in 1115 bytes} [Wed Aug 24 11:13:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:13.007204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca142a-f01b-4e90-b971-8314653c876e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:13.114275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca142a-f01b-4e90-b971-8314653c876e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:13.150138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519b483-6356-476c-9a84-a429b1840db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:13.150758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519b483-6356-476c-9a84-a429b1840db4 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:13.151036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519b483-6356-476c-9a84-a429b1840db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:13.151306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519b483-6356-476c-9a84-a429b1840db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:13.161198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519b483-6356-476c-9a84-a429b1840db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:13.162631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519b483-6356-476c-9a84-a429b1840db4 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-9519b483-6356-476c-9a84-a429b1840db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:13:13.177349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:13.178411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:13:13.180202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:13:13.180202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:13.180202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:13.180949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 540/1089] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:13:13.185204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c3f46-15a6-4299-8886-0f8272ca38d3 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:13.283136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c3f46-15a6-4299-8886-0f8272ca38d3 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:13.313292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:13.318117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d7196-8eab-4d77-ab97-ae03fbf1fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:13.318719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d7196-8eab-4d77-ab97-ae03fbf1fe89 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:13.321468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d7196-8eab-4d77-ab97-ae03fbf1fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:13.321468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d7196-8eab-4d77-ab97-ae03fbf1fe89 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:13.321468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d7196-8eab-4d77-ab97-ae03fbf1fe89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:13.321468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d7196-8eab-4d77-ab97-ae03fbf1fe89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:13.322094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d7196-8eab-4d77-ab97-ae03fbf1fe89 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:13.333624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 550/1090] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:13.339351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519b483-6356-476c-9a84-a429b1840db4 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:13.389320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:13.441385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519b483-6356-476c-9a84-a429b1840db4 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:13.442807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691c463-13c9-4574-ba50-9048705bd77a None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:13.443256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691c463-13c9-4574-ba50-9048705bd77a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:13.443360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691c463-13c9-4574-ba50-9048705bd77a None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:13.444691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 551/1091] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:13:13.450331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac2009-7db7-4a51-b8e2-69333027575c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:13.450616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac2009-7db7-4a51-b8e2-69333027575c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:13.451248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac2009-7db7-4a51-b8e2-69333027575c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:13.468541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-afac2009-7db7-4a51-b8e2-69333027575c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:13.469763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-afac2009-7db7-4a51-b8e2-69333027575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:13.476760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb340c-d0c3-4b3d-9f17-583c3f475f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:13.477413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb340c-d0c3-4b3d-9f17-583c3f475f90 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:13.479511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb340c-d0c3-4b3d-9f17-583c3f475f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:13.479511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb340c-d0c3-4b3d-9f17-583c3f475f90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:13.485980 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb340c-d0c3-4b3d-9f17-583c3f475f90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:13.487486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb340c-d0c3-4b3d-9f17-583c3f475f90 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-28eb340c-d0c3-4b3d-9f17-583c3f475f90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:13:13.499063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:13.500079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:13.501149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 541/1092] 10.222.0.84 () {56 vars in 1093 bytes} [Wed Aug 24 11:13:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:13:13.505880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb340c-d0c3-4b3d-9f17-583c3f475f90 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:13.571730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 552/1093] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 11:13:13.604254 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb340c-d0c3-4b3d-9f17-583c3f475f90 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:13.613724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:13.638332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9297e18-0e2d-4dd8-8651-44d9fad8cda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:13.638923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9297e18-0e2d-4dd8-8651-44d9fad8cda2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:13.639196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9297e18-0e2d-4dd8-8651-44d9fad8cda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:13.639463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9297e18-0e2d-4dd8-8651-44d9fad8cda2 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:13.640169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9297e18-0e2d-4dd8-8651-44d9fad8cda2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:13.640467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9297e18-0e2d-4dd8-8651-44d9fad8cda2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:13.641895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9297e18-0e2d-4dd8-8651-44d9fad8cda2 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:13.659557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 542/1094] 10.222.0.84 () {56 vars in 1133 bytes} [Wed Aug 24 11:13:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:13.665834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac2009-7db7-4a51-b8e2-69333027575c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:13.763587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac2009-7db7-4a51-b8e2-69333027575c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:13.785794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:13.797130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba75376-5a4e-4d77-9bde-d3cca18ad423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:13.797724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba75376-5a4e-4d77-9bde-d3cca18ad423 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:13.797994 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba75376-5a4e-4d77-9bde-d3cca18ad423 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:13.798258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba75376-5a4e-4d77-9bde-d3cca18ad423 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:13.848939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9297e18-0e2d-4dd8-8651-44d9fad8cda2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:13.863856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba75376-5a4e-4d77-9bde-d3cca18ad423 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:13.865319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba75376-5a4e-4d77-9bde-d3cca18ad423 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:13.947966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9297e18-0e2d-4dd8-8651-44d9fad8cda2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:13.968119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 553/1095] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:13:13.969313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba75376-5a4e-4d77-9bde-d3cca18ad423 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:13.982006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de51a8-6e23-45f4-90e2-c11575f97c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:13.982641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de51a8-6e23-45f4-90e2-c11575f97c73 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:13.982914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de51a8-6e23-45f4-90e2-c11575f97c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:13.983177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de51a8-6e23-45f4-90e2-c11575f97c73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:13.992657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:13.993633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de51a8-6e23-45f4-90e2-c11575f97c73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e1de51a8-6e23-45f4-90e2-c11575f97c73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:13.998308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:13.999251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:14.000069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 543/1096] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:13:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:13:14.005420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de51a8-6e23-45f4-90e2-c11575f97c73 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:14.034307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:14.067334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba75376-5a4e-4d77-9bde-d3cca18ad423 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.101867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f501-4955-491d-b27f-67254d122b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.102486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f501-4955-491d-b27f-67254d122b97 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.102760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f501-4955-491d-b27f-67254d122b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.103030 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f501-4955-491d-b27f-67254d122b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.105816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de51a8-6e23-45f4-90e2-c11575f97c73 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.113025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0f501-4955-491d-b27f-67254d122b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.114454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0f501-4955-491d-b27f-67254d122b97 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-fab0f501-4955-491d-b27f-67254d122b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:13:14.128459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:14.129626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:13:14.130532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:13:14.130532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:13:14.130532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:14.130532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 554/1097] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:14.135795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f501-4955-491d-b27f-67254d122b97 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:14.142164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646b793-6283-43cc-9ea5-483378028291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.142763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646b793-6283-43cc-9ea5-483378028291 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.143039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646b793-6283-43cc-9ea5-483378028291 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.143307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646b793-6283-43cc-9ea5-483378028291 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.143959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646b793-6283-43cc-9ea5-483378028291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:14.144260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646b793-6283-43cc-9ea5-483378028291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.145640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646b793-6283-43cc-9ea5-483378028291 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.154398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 544/1098] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:13:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:14.160081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646b793-6283-43cc-9ea5-483378028291 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:14.209346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:14.233783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f501-4955-491d-b27f-67254d122b97 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.246729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:14.259201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646b793-6283-43cc-9ea5-483378028291 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.267205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783664b6-7938-4d0c-9eb6-ef9792c3b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.267806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783664b6-7938-4d0c-9eb6-ef9792c3b267 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.268080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783664b6-7938-4d0c-9eb6-ef9792c3b267 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.268347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783664b6-7938-4d0c-9eb6-ef9792c3b267 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.269014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783664b6-7938-4d0c-9eb6-ef9792c3b267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:14.269313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783664b6-7938-4d0c-9eb6-ef9792c3b267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.270667 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783664b6-7938-4d0c-9eb6-ef9792c3b267 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.283132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 555/1099] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:14.288926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783664b6-7938-4d0c-9eb6-ef9792c3b267 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:14.295869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbba20-3082-428c-8f41-d2ad3f990f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.296492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbba20-3082-428c-8f41-d2ad3f990f87 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.297193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbba20-3082-428c-8f41-d2ad3f990f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.297471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbba20-3082-428c-8f41-d2ad3f990f87 None admin] PATH_INFO: `/v3/users/28cd3a5305e442a690322c76d5fbfbc1` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.298593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbba20-3082-428c-8f41-d2ad3f990f87 None admin] RBAC: Authorizing `identity:get_user(user_id=28cd3a5305e442a690322c76d5fbfbc1)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.300654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbba20-3082-428c-8f41-d2ad3f990f87 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.307335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 545/1100] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:14 2022] GET /identity/v3/users/28cd3a5305e442a690322c76d5fbfbc1 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:14.308285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbba20-3082-428c-8f41-d2ad3f990f87 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:14.387447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783664b6-7938-4d0c-9eb6-ef9792c3b267 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.407405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbba20-3082-428c-8f41-d2ad3f990f87 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.420445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a2b84-250c-4dcf-85d1-dbaeccea031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.421089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a2b84-250c-4dcf-85d1-dbaeccea031d None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.421362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a2b84-250c-4dcf-85d1-dbaeccea031d None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.421625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a2b84-250c-4dcf-85d1-dbaeccea031d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.423455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:14.429141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a2b84-250c-4dcf-85d1-dbaeccea031d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.430580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a2b84-250c-4dcf-85d1-dbaeccea031d None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-cd1a2b84-250c-4dcf-85d1-dbaeccea031d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:13:14.441392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:14.442454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:14.443757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:13:14.443757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:13:14.443757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:13:14.443757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:13:14.443757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:13:14.443757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:13:14.443757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:13:14.443757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:14.444890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0367610-3d66-41a1-91b9-acfb71e5b658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.444890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0367610-3d66-41a1-91b9-acfb71e5b658 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.444890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0367610-3d66-41a1-91b9-acfb71e5b658 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.445107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 556/1101] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:13:14.445178 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0367610-3d66-41a1-91b9-acfb71e5b658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.448872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a2b84-250c-4dcf-85d1-dbaeccea031d None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:14.468160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:14.511196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0367610-3d66-41a1-91b9-acfb71e5b658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.512632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0367610-3d66-41a1-91b9-acfb71e5b658 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.550267 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a2b84-250c-4dcf-85d1-dbaeccea031d None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.587413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e069c-4f03-4f36-9418-477787c7633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.588018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e069c-4f03-4f36-9418-477787c7633a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.588269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e069c-4f03-4f36-9418-477787c7633a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.588525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e069c-4f03-4f36-9418-477787c7633a None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.589222 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e069c-4f03-4f36-9418-477787c7633a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:14.589515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e069c-4f03-4f36-9418-477787c7633a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.590916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e069c-4f03-4f36-9418-477787c7633a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.603315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 557/1102] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:13:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:14.609595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e069c-4f03-4f36-9418-477787c7633a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:14.618944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 546/1103] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:13:14.624771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0367610-3d66-41a1-91b9-acfb71e5b658 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:14.664745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:14.684039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:14.711295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e069c-4f03-4f36-9418-477787c7633a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.727175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0367610-3d66-41a1-91b9-acfb71e5b658 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.745074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cd480-4dba-48e3-8b0b-85d2599ad83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.745670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cd480-4dba-48e3-8b0b-85d2599ad83a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.745940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cd480-4dba-48e3-8b0b-85d2599ad83a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.746202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cd480-4dba-48e3-8b0b-85d2599ad83a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.746903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3cd480-4dba-48e3-8b0b-85d2599ad83a None admin] RBAC: Adding query filter params (role.id=b2da7bb417d94f45a2a4e8a2befa0988, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=f58a559951c94d68951da78c82341a00) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:14.747199 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3cd480-4dba-48e3-8b0b-85d2599ad83a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.748598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3cd480-4dba-48e3-8b0b-85d2599ad83a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.754460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 558/1104] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:13:14 2022] GET /identity/v3/role_assignments?user.id=f58a559951c94d68951da78c82341a00&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=b2da7bb417d94f45a2a4e8a2befa0988 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:14.762513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc5cec-9632-4115-a33b-a22c5fb23218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.763256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc5cec-9632-4115-a33b-a22c5fb23218 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.763603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc5cec-9632-4115-a33b-a22c5fb23218 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.763965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc5cec-9632-4115-a33b-a22c5fb23218 None admin] PATH_INFO: `/v3/projects/286f0726633b449d936b1774cc84e8dc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.765273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc5cec-9632-4115-a33b-a22c5fb23218 None admin] RBAC: Authorizing `identity:get_project(project_id=286f0726633b449d936b1774cc84e8dc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.766867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc5cec-9632-4115-a33b-a22c5fb23218 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.768682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 547/1105] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:14 2022] GET /identity/v3/projects/286f0726633b449d936b1774cc84e8dc => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:14.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cd480-4dba-48e3-8b0b-85d2599ad83a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:14.885473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cd480-4dba-48e3-8b0b-85d2599ad83a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:14.907262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:14.924575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec9c55-5d65-44b0-ad3f-676db69ef5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:14.925241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec9c55-5d65-44b0-ad3f-676db69ef5bb None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:14.925522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec9c55-5d65-44b0-ad3f-676db69ef5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:14.925782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec9c55-5d65-44b0-ad3f-676db69ef5bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:14.926639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec9c55-5d65-44b0-ad3f-676db69ef5bb None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=286f0726633b449d936b1774cc84e8dc, user.id=28cd3a5305e442a690322c76d5fbfbc1) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:14.926973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec9c55-5d65-44b0-ad3f-676db69ef5bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:14.928420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec9c55-5d65-44b0-ad3f-676db69ef5bb None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:14.935130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 559/1106] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:13:14 2022] GET /identity/v3/role_assignments?user.id=28cd3a5305e442a690322c76d5fbfbc1&scope.project.id=286f0726633b449d936b1774cc84e8dc&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:16.873859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b526d-8d36-4f38-9b47-38873b62c1a6 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:16.874387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b526d-8d36-4f38-9b47-38873b62c1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:16.874387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b526d-8d36-4f38-9b47-38873b62c1a6 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:16.875828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 548/1107] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:16.880986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d1eda-1014-4a33-9a6f-995c59e7a4d9 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:16.881389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d1eda-1014-4a33-9a6f-995c59e7a4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:16.881562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d1eda-1014-4a33-9a6f-995c59e7a4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:16.899598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-995d1eda-1014-4a33-9a6f-995c59e7a4d9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:16.900757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-995d1eda-1014-4a33-9a6f-995c59e7a4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:16.969814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 560/1108] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:17.236360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b526d-8d36-4f38-9b47-38873b62c1a6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:17.339028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b526d-8d36-4f38-9b47-38873b62c1a6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:17.362165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:17.372706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbd54f-04ed-41aa-a297-a40f6a7af84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:17.373346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd54f-04ed-41aa-a297-a40f6a7af84b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:17.373621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd54f-04ed-41aa-a297-a40f6a7af84b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:17.373889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd54f-04ed-41aa-a297-a40f6a7af84b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:17.374984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbd54f-04ed-41aa-a297-a40f6a7af84b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:17.376392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbd54f-04ed-41aa-a297-a40f6a7af84b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:17.377973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 549/1109] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:17.384068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d1eda-1014-4a33-9a6f-995c59e7a4d9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:17.480490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d1eda-1014-4a33-9a6f-995c59e7a4d9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:17.494056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:17.514266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fef3af-4cb3-467d-b20d-ed142e95f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:17.514854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fef3af-4cb3-467d-b20d-ed142e95f697 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:17.515125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fef3af-4cb3-467d-b20d-ed142e95f697 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:17.515443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fef3af-4cb3-467d-b20d-ed142e95f697 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:17.516104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fef3af-4cb3-467d-b20d-ed142e95f697 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:17.517541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fef3af-4cb3-467d-b20d-ed142e95f697 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:17.660217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 561/1110] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:13:17 2022] POST /identity/v3/users => generated 286 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:13:19.573852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993b66e-1b98-4dab-8178-34f466ef35ca None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:19.574394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993b66e-1b98-4dab-8178-34f466ef35ca None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:19.574394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993b66e-1b98-4dab-8178-34f466ef35ca None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:19.575774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 550/1111] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:19.581123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7bb25-cda1-4a8d-abd5-1a08cefb4d38 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:19.581587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7bb25-cda1-4a8d-abd5-1a08cefb4d38 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:19.581682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7bb25-cda1-4a8d-abd5-1a08cefb4d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:19.600104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-49c7bb25-cda1-4a8d-abd5-1a08cefb4d38 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:19.601515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-49c7bb25-cda1-4a8d-abd5-1a08cefb4d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:19.667393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 562/1112] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:19.936367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993b66e-1b98-4dab-8178-34f466ef35ca None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:20.036572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993b66e-1b98-4dab-8178-34f466ef35ca None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:20.070151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fdf20-c2ef-41b6-87e4-9a459d8bafd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:20.070765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fdf20-c2ef-41b6-87e4-9a459d8bafd3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:20.071045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fdf20-c2ef-41b6-87e4-9a459d8bafd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:20.071319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fdf20-c2ef-41b6-87e4-9a459d8bafd3 None admin] PATH_INFO: `/v3/roles/1fbe1ab7a2864ba1818c7038baf57c34` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:20.072704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fdf20-c2ef-41b6-87e4-9a459d8bafd3 None admin] RBAC: Authorizing `identity:get_role(role_id=1fbe1ab7a2864ba1818c7038baf57c34)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:20.074084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fdf20-c2ef-41b6-87e4-9a459d8bafd3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:20.075391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 551/1113] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:19 2022] GET /identity/v3/roles/1fbe1ab7a2864ba1818c7038baf57c34 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:20.081352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7bb25-cda1-4a8d-abd5-1a08cefb4d38 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:20.173768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:20.177256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7bb25-cda1-4a8d-abd5-1a08cefb4d38 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:20.212546 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da2c08-35dc-438c-84ea-e58d7ac57653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:20.213135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da2c08-35dc-438c-84ea-e58d7ac57653 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:20.213614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da2c08-35dc-438c-84ea-e58d7ac57653 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:20.214102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da2c08-35dc-438c-84ea-e58d7ac57653 None admin] PATH_INFO: `/v3/users/d64705b54c6e40108995da8c079bc46c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:20.227701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da2c08-35dc-438c-84ea-e58d7ac57653 None admin] RBAC: Authorizing `identity:get_user(user_id=d64705b54c6e40108995da8c079bc46c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:20.231014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da2c08-35dc-438c-84ea-e58d7ac57653 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:20.231014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 563/1114] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:20 2022] GET /identity/v3/users/d64705b54c6e40108995da8c079bc46c => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:20.237222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fdf20-c2ef-41b6-87e4-9a459d8bafd3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:20.295451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:20.334749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fdf20-c2ef-41b6-87e4-9a459d8bafd3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:20.368317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89081674-dce8-461f-85b1-21f3c5015a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:20.368934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89081674-dce8-461f-85b1-21f3c5015a4e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:20.369210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89081674-dce8-461f-85b1-21f3c5015a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:20.369477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89081674-dce8-461f-85b1-21f3c5015a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:20.433642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89081674-dce8-461f-85b1-21f3c5015a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:20.435068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89081674-dce8-461f-85b1-21f3c5015a4e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:20.490405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:20.529508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 552/1115] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:13:20.535539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da2c08-35dc-438c-84ea-e58d7ac57653 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:20.693908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da2c08-35dc-438c-84ea-e58d7ac57653 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:20.727443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df4b73-5d76-4f30-8f45-678ae040eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:20.728055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df4b73-5d76-4f30-8f45-678ae040eb0c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:20.728371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df4b73-5d76-4f30-8f45-678ae040eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:20.728640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df4b73-5d76-4f30-8f45-678ae040eb0c None admin] PATH_INFO: `/v3/projects/286f0726633b449d936b1774cc84e8dc` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:20.729765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df4b73-5d76-4f30-8f45-678ae040eb0c None admin] RBAC: Authorizing `identity:get_project(project_id=286f0726633b449d936b1774cc84e8dc)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:20.731220 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df4b73-5d76-4f30-8f45-678ae040eb0c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:20.733031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 564/1116] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:20 2022] GET /identity/v3/projects/286f0726633b449d936b1774cc84e8dc => generated 311 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:20.739301 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89081674-dce8-461f-85b1-21f3c5015a4e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:20.836631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89081674-dce8-461f-85b1-21f3c5015a4e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:20.869953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e0799-ffcf-4429-9308-2b826f48d65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:20.870543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e0799-ffcf-4429-9308-2b826f48d65e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:20.870822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e0799-ffcf-4429-9308-2b826f48d65e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:20.871094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e0799-ffcf-4429-9308-2b826f48d65e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:20.871805 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e0799-ffcf-4429-9308-2b826f48d65e None admin] RBAC: Adding query filter params (role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=286f0726633b449d936b1774cc84e8dc, user.id=d64705b54c6e40108995da8c079bc46c) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:20.872106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e0799-ffcf-4429-9308-2b826f48d65e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:20.873548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e0799-ffcf-4429-9308-2b826f48d65e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:20.878800 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 553/1117] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:13:20 2022] GET /identity/v3/role_assignments?user.id=d64705b54c6e40108995da8c079bc46c&scope.project.id=286f0726633b449d936b1774cc84e8dc&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:22.742968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601812e5-5bd6-4da3-a8a9-d6737d077942 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:22.743488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601812e5-5bd6-4da3-a8a9-d6737d077942 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:22.743488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601812e5-5bd6-4da3-a8a9-d6737d077942 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:22.744871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 565/1118] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:22.749948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80ab9b-223a-4777-a890-9cb57cd2e35a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:22.750349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80ab9b-223a-4777-a890-9cb57cd2e35a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:22.750492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80ab9b-223a-4777-a890-9cb57cd2e35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:22.767998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9f80ab9b-223a-4777-a890-9cb57cd2e35a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:22.769208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9f80ab9b-223a-4777-a890-9cb57cd2e35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:22.786422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:22.834043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 554/1119] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:23.100099 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601812e5-5bd6-4da3-a8a9-d6737d077942 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:23.124911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:23.198510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601812e5-5bd6-4da3-a8a9-d6737d077942 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:23.230990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd237f21-5628-4919-b398-48b9c605e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:23.231600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd237f21-5628-4919-b398-48b9c605e824 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:23.231877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd237f21-5628-4919-b398-48b9c605e824 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:23.232144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd237f21-5628-4919-b398-48b9c605e824 None admin] PATH_INFO: `/v3/roles/1fbe1ab7a2864ba1818c7038baf57c34` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:23.233780 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd237f21-5628-4919-b398-48b9c605e824 None admin] RBAC: Authorizing `identity:get_role(role_id=1fbe1ab7a2864ba1818c7038baf57c34)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:23.235133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd237f21-5628-4919-b398-48b9c605e824 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:23.236443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 566/1120] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:23 2022] GET /identity/v3/roles/1fbe1ab7a2864ba1818c7038baf57c34 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:23.242337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80ab9b-223a-4777-a890-9cb57cd2e35a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:23.342696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80ab9b-223a-4777-a890-9cb57cd2e35a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:23.375766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3b2fe-df16-4a3b-9531-2c22a67fb03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:23.376320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3b2fe-df16-4a3b-9531-2c22a67fb03b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:23.376595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3b2fe-df16-4a3b-9531-2c22a67fb03b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:23.376892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3b2fe-df16-4a3b-9531-2c22a67fb03b None admin] PATH_INFO: `/v3/users/d64705b54c6e40108995da8c079bc46c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:23.377984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3b2fe-df16-4a3b-9531-2c22a67fb03b None admin] RBAC: Authorizing `identity:get_user(user_id=d64705b54c6e40108995da8c079bc46c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:23.379409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3b2fe-df16-4a3b-9531-2c22a67fb03b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:23.380997 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 555/1121] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:23 2022] GET /identity/v3/users/d64705b54c6e40108995da8c079bc46c => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:23.387829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd237f21-5628-4919-b398-48b9c605e824 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:23.475190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:23.484006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd237f21-5628-4919-b398-48b9c605e824 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:23.517926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1832760-edf2-4a86-bda3-117afafad7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:23.518535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832760-edf2-4a86-bda3-117afafad7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:23.518807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832760-edf2-4a86-bda3-117afafad7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:23.519069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832760-edf2-4a86-bda3-117afafad7f9 None admin] PATH_INFO: `/v3/projects/286f0726633b449d936b1774cc84e8dc` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:23.520149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1832760-edf2-4a86-bda3-117afafad7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=286f0726633b449d936b1774cc84e8dc)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:23.521572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1832760-edf2-4a86-bda3-117afafad7f9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:23.523239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 567/1122] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:23 2022] GET /identity/v3/projects/286f0726633b449d936b1774cc84e8dc => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:23.529171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3b2fe-df16-4a3b-9531-2c22a67fb03b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:23.552789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:23.626068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3b2fe-df16-4a3b-9531-2c22a67fb03b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:23.660740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dd44a-17dc-4fe2-9155-2913e8086926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:23.661415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dd44a-17dc-4fe2-9155-2913e8086926 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:23.661689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dd44a-17dc-4fe2-9155-2913e8086926 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:23.661957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dd44a-17dc-4fe2-9155-2913e8086926 None admin] PATH_INFO: `/v3/projects/286f0726633b449d936b1774cc84e8dc/users/d64705b54c6e40108995da8c079bc46c/roles/1fbe1ab7a2864ba1818c7038baf57c34` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:23.664072 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dd44a-17dc-4fe2-9155-2913e8086926 None admin] RBAC: Authorizing `identity:create_grant(project_id=286f0726633b449d936b1774cc84e8dc, user_id=d64705b54c6e40108995da8c079bc46c, role_id=1fbe1ab7a2864ba1818c7038baf57c34)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:23.665579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dd44a-17dc-4fe2-9155-2913e8086926 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:23.739568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 556/1123] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:13:23 2022] PUT /identity/v3/projects/286f0726633b449d936b1774cc84e8dc/users/d64705b54c6e40108995da8c079bc46c/roles/1fbe1ab7a2864ba1818c7038baf57c34 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:13:25.615351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886e33f-d738-4ec5-ba59-83bdd868b912 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:25.615985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886e33f-d738-4ec5-ba59-83bdd868b912 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:25.615985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886e33f-d738-4ec5-ba59-83bdd868b912 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:25.617854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 568/1124] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:25.622974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12fb86-b879-4c90-b4c2-8cf7b7ff2930 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:25.623355 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12fb86-b879-4c90-b4c2-8cf7b7ff2930 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:25.623521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12fb86-b879-4c90-b4c2-8cf7b7ff2930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:25.641130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6a12fb86-b879-4c90-b4c2-8cf7b7ff2930 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:25.642294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6a12fb86-b879-4c90-b4c2-8cf7b7ff2930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:25.712164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 557/1125] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:25.979263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886e33f-d738-4ec5-ba59-83bdd868b912 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:26.077436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886e33f-d738-4ec5-ba59-83bdd868b912 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:26.110355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8e3a8-234a-49d9-a25c-7a37947351f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:26.110957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8e3a8-234a-49d9-a25c-7a37947351f7 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:26.111228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8e3a8-234a-49d9-a25c-7a37947351f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:26.111491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8e3a8-234a-49d9-a25c-7a37947351f7 None admin] PATH_INFO: `/v3/roles/1fbe1ab7a2864ba1818c7038baf57c34` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:26.112845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8e3a8-234a-49d9-a25c-7a37947351f7 None admin] RBAC: Authorizing `identity:get_role(role_id=1fbe1ab7a2864ba1818c7038baf57c34)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:26.114187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8e3a8-234a-49d9-a25c-7a37947351f7 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:26.115467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 569/1126] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:25 2022] GET /identity/v3/roles/1fbe1ab7a2864ba1818c7038baf57c34 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:26.121264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12fb86-b879-4c90-b4c2-8cf7b7ff2930 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:26.135804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:26.217845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12fb86-b879-4c90-b4c2-8cf7b7ff2930 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:26.250585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c0c75-d80e-4d5f-9189-b7169c83664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:26.251135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c0c75-d80e-4d5f-9189-b7169c83664c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:26.251411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c0c75-d80e-4d5f-9189-b7169c83664c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:26.251680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c0c75-d80e-4d5f-9189-b7169c83664c None admin] PATH_INFO: `/v3/users/d64705b54c6e40108995da8c079bc46c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:26.252777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c0c75-d80e-4d5f-9189-b7169c83664c None admin] RBAC: Authorizing `identity:get_user(user_id=d64705b54c6e40108995da8c079bc46c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:26.254226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c0c75-d80e-4d5f-9189-b7169c83664c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:26.255782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 558/1127] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:26 2022] GET /identity/v3/users/d64705b54c6e40108995da8c079bc46c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:26.262080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8e3a8-234a-49d9-a25c-7a37947351f7 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:26.289715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:26.359367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8e3a8-234a-49d9-a25c-7a37947351f7 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:26.392193 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbf043-62b4-488a-bce4-f6231b54a713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:26.392775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbf043-62b4-488a-bce4-f6231b54a713 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:26.393074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbf043-62b4-488a-bce4-f6231b54a713 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:26.393333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbf043-62b4-488a-bce4-f6231b54a713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:26.456817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbf043-62b4-488a-bce4-f6231b54a713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:26.458404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbf043-62b4-488a-bce4-f6231b54a713 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:26.483557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:26.552247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 570/1128] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:13:26.558842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c0c75-d80e-4d5f-9189-b7169c83664c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:26.637883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:26.656638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c0c75-d80e-4d5f-9189-b7169c83664c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:26.691719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbda630-3c33-4489-8dca-b2e471764f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:26.692338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbda630-3c33-4489-8dca-b2e471764f2e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:26.692616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbda630-3c33-4489-8dca-b2e471764f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:26.692918 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbda630-3c33-4489-8dca-b2e471764f2e None admin] PATH_INFO: `/v3/projects/286f0726633b449d936b1774cc84e8dc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:26.694579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbda630-3c33-4489-8dca-b2e471764f2e None admin] RBAC: Authorizing `identity:get_project(project_id=286f0726633b449d936b1774cc84e8dc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:26.696001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbda630-3c33-4489-8dca-b2e471764f2e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:26.697655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 559/1129] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:26 2022] GET /identity/v3/projects/286f0726633b449d936b1774cc84e8dc => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:26.703765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbf043-62b4-488a-bce4-f6231b54a713 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:26.799954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbf043-62b4-488a-bce4-f6231b54a713 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:26.825454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:26.832782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff89ad8-570e-4b6b-8fe0-a8a3a83ae296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:26.835790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89ad8-570e-4b6b-8fe0-a8a3a83ae296 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:26.835790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89ad8-570e-4b6b-8fe0-a8a3a83ae296 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:26.835790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff89ad8-570e-4b6b-8fe0-a8a3a83ae296 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:26.835790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89ad8-570e-4b6b-8fe0-a8a3a83ae296 None admin] RBAC: Adding query filter params (role.id=1fbe1ab7a2864ba1818c7038baf57c34, scope.project.id=286f0726633b449d936b1774cc84e8dc, user.id=d64705b54c6e40108995da8c079bc46c) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:26.835790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89ad8-570e-4b6b-8fe0-a8a3a83ae296 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:26.836560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff89ad8-570e-4b6b-8fe0-a8a3a83ae296 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:26.842125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 571/1130] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:13:26 2022] GET /identity/v3/role_assignments?user.id=d64705b54c6e40108995da8c079bc46c&scope.project.id=286f0726633b449d936b1774cc84e8dc&role.id=1fbe1ab7a2864ba1818c7038baf57c34 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:28.815197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade0015-23e5-40e4-9c0d-439d20da99db None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:28.815779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade0015-23e5-40e4-9c0d-439d20da99db None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:28.815779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade0015-23e5-40e4-9c0d-439d20da99db None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:28.817107 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 560/1131] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:28.822838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb161dd-efe6-467e-8cc7-ff6d23e5b940 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:28.823241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb161dd-efe6-467e-8cc7-ff6d23e5b940 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:28.823379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb161dd-efe6-467e-8cc7-ff6d23e5b940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:28.841482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-dcb161dd-efe6-467e-8cc7-ff6d23e5b940 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:28.842660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-dcb161dd-efe6-467e-8cc7-ff6d23e5b940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:28.909025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 572/1132] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:29.182676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade0015-23e5-40e4-9c0d-439d20da99db None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:29.280842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade0015-23e5-40e4-9c0d-439d20da99db None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:29.315834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93dd7be-8339-444c-88f4-c4edcf4ac9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:29.316434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dd7be-8339-444c-88f4-c4edcf4ac9ff None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:29.316704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dd7be-8339-444c-88f4-c4edcf4ac9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:29.316994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93dd7be-8339-444c-88f4-c4edcf4ac9ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:29.318071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93dd7be-8339-444c-88f4-c4edcf4ac9ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:29.319638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93dd7be-8339-444c-88f4-c4edcf4ac9ff None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:29.321205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 561/1133] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:29.326936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb161dd-efe6-467e-8cc7-ff6d23e5b940 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:29.422091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb161dd-efe6-467e-8cc7-ff6d23e5b940 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:29.443340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:29.453995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa3d3d-3791-482d-92c4-019c13dda8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:29.454579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa3d3d-3791-482d-92c4-019c13dda8af None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:29.456862 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa3d3d-3791-482d-92c4-019c13dda8af None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:29.456862 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa3d3d-3791-482d-92c4-019c13dda8af None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:29.456862 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa3d3d-3791-482d-92c4-019c13dda8af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:29.457115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa3d3d-3791-482d-92c4-019c13dda8af None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:29.587100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 573/1134] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:13:29 2022] POST /identity/v3/projects => generated 311 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:13:31.589717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a8a3-582c-4ac8-96c5-f4648cb9f181 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:31.590284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a8a3-582c-4ac8-96c5-f4648cb9f181 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:31.590284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057a8a3-582c-4ac8-96c5-f4648cb9f181 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:31.591572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 562/1135] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:31.596988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf456124-b84d-4577-a498-a339bf39a8f4 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:31.597427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf456124-b84d-4577-a498-a339bf39a8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:31.597532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf456124-b84d-4577-a498-a339bf39a8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:31.616904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-bf456124-b84d-4577-a498-a339bf39a8f4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:31.618069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-bf456124-b84d-4577-a498-a339bf39a8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:31.681931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 574/1136] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:31.949444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a8a3-582c-4ac8-96c5-f4648cb9f181 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:31.968527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:32.051146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057a8a3-582c-4ac8-96c5-f4648cb9f181 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:32.083941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17aea16-44d4-434d-9696-331ec52bdf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:32.085182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17aea16-44d4-434d-9696-331ec52bdf08 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:32.085454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17aea16-44d4-434d-9696-331ec52bdf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:32.085722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17aea16-44d4-434d-9696-331ec52bdf08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:32.086809 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17aea16-44d4-434d-9696-331ec52bdf08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:32.088278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17aea16-44d4-434d-9696-331ec52bdf08 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:32.089845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 563/1137] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:13:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:32.095643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf456124-b84d-4577-a498-a339bf39a8f4 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:32.190389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf456124-b84d-4577-a498-a339bf39a8f4 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:32.192606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:32.223543 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cde30-64c1-4b4b-99fd-ef1cd698132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:32.224073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cde30-64c1-4b4b-99fd-ef1cd698132f None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:32.224347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cde30-64c1-4b4b-99fd-ef1cd698132f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:32.224614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cde30-64c1-4b4b-99fd-ef1cd698132f None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:32.225304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cde30-64c1-4b4b-99fd-ef1cd698132f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:32.226715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cde30-64c1-4b4b-99fd-ef1cd698132f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:32.413759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 575/1138] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:13:32 2022] POST /identity/v3/users => generated 286 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:13:34.377352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde925f-b3e7-4551-b178-e5a4087fd11f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:34.377900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde925f-b3e7-4551-b178-e5a4087fd11f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:34.377900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde925f-b3e7-4551-b178-e5a4087fd11f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:34.379199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 564/1139] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:34.384942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1a3d6-d4fb-4c78-b4f7-600fbf259f1f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:34.385343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1a3d6-d4fb-4c78-b4f7-600fbf259f1f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:34.385492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1a3d6-d4fb-4c78-b4f7-600fbf259f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:34.403592 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c8b1a3d6-d4fb-4c78-b4f7-600fbf259f1f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:34.404779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c8b1a3d6-d4fb-4c78-b4f7-600fbf259f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:34.468791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 576/1140] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:34.740106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde925f-b3e7-4551-b178-e5a4087fd11f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:34.823297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:34.840674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde925f-b3e7-4551-b178-e5a4087fd11f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:34.874944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72219985-96a1-4cab-9f2e-6f2cc899b468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:34.875550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72219985-96a1-4cab-9f2e-6f2cc899b468 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:34.875823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72219985-96a1-4cab-9f2e-6f2cc899b468 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:34.876091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72219985-96a1-4cab-9f2e-6f2cc899b468 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:34.886453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72219985-96a1-4cab-9f2e-6f2cc899b468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-72219985-96a1-4cab-9f2e-6f2cc899b468 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:34.891017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:34.892164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:34.893288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 565/1141] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:13:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:34.898325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1a3d6-d4fb-4c78-b4f7-600fbf259f1f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:34.985965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:34.995023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1a3d6-d4fb-4c78-b4f7-600fbf259f1f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:35.028404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06385f-e021-4569-98cb-fdda0c4c07ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:35.029029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06385f-e021-4569-98cb-fdda0c4c07ca None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:35.029307 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06385f-e021-4569-98cb-fdda0c4c07ca None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:35.029575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06385f-e021-4569-98cb-fdda0c4c07ca None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:35.030227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06385f-e021-4569-98cb-fdda0c4c07ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:35.030526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06385f-e021-4569-98cb-fdda0c4c07ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:35.031878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06385f-e021-4569-98cb-fdda0c4c07ca None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:35.039740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 577/1142] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:13:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:35.045548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72219985-96a1-4cab-9f2e-6f2cc899b468 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:35.158066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72219985-96a1-4cab-9f2e-6f2cc899b468 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:35.197863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85901d83-6701-4535-bb18-79a5972f1a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:35.198673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85901d83-6701-4535-bb18-79a5972f1a32 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:35.199086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85901d83-6701-4535-bb18-79a5972f1a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:35.199492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85901d83-6701-4535-bb18-79a5972f1a32 None admin] PATH_INFO: `/v3/users/543daa9b358b4460a9214ed2004c0ece` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:35.218424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85901d83-6701-4535-bb18-79a5972f1a32 None admin] RBAC: Authorizing `identity:get_user(user_id=543daa9b358b4460a9214ed2004c0ece)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:35.219874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85901d83-6701-4535-bb18-79a5972f1a32 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:35.221566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 566/1143] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:35 2022] GET /identity/v3/users/543daa9b358b4460a9214ed2004c0ece => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:35.228214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06385f-e021-4569-98cb-fdda0c4c07ca None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:35.327910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06385f-e021-4569-98cb-fdda0c4c07ca None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:35.360990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23b328-4d4d-4d61-9268-f998cdb86778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:35.361581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23b328-4d4d-4d61-9268-f998cdb86778 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:35.361856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23b328-4d4d-4d61-9268-f998cdb86778 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:35.362123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23b328-4d4d-4d61-9268-f998cdb86778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:35.383470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:35.426966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23b328-4d4d-4d61-9268-f998cdb86778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:35.428934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23b328-4d4d-4d61-9268-f998cdb86778 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:35.536237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 578/1144] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:13:35.542285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85901d83-6701-4535-bb18-79a5972f1a32 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:35.545141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:35.642465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85901d83-6701-4535-bb18-79a5972f1a32 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:35.674702 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495354f-7ae6-4216-8599-3553e18295b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:35.675315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495354f-7ae6-4216-8599-3553e18295b6 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:35.675586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495354f-7ae6-4216-8599-3553e18295b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:35.675852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495354f-7ae6-4216-8599-3553e18295b6 None admin] PATH_INFO: `/v3/projects/dee508628d3e4a5da2765185fe58ed11` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:35.676961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495354f-7ae6-4216-8599-3553e18295b6 None admin] RBAC: Authorizing `identity:get_project(project_id=dee508628d3e4a5da2765185fe58ed11)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:35.679044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495354f-7ae6-4216-8599-3553e18295b6 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:35.680647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 567/1145] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:35 2022] GET /identity/v3/projects/dee508628d3e4a5da2765185fe58ed11 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:35.686723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23b328-4d4d-4d61-9268-f998cdb86778 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:35.737935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:35.783427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23b328-4d4d-4d61-9268-f998cdb86778 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:35.816371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cb9b9-6b53-488b-ab6b-cad6b9375de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:35.816980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cb9b9-6b53-488b-ab6b-cad6b9375de4 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:35.817254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cb9b9-6b53-488b-ab6b-cad6b9375de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:35.817517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cb9b9-6b53-488b-ab6b-cad6b9375de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:35.818214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cb9b9-6b53-488b-ab6b-cad6b9375de4 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=dee508628d3e4a5da2765185fe58ed11, user.id=543daa9b358b4460a9214ed2004c0ece) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:35.818591 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cb9b9-6b53-488b-ab6b-cad6b9375de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:35.820077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cb9b9-6b53-488b-ab6b-cad6b9375de4 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:35.825212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 579/1146] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:13:35 2022] GET /identity/v3/role_assignments?user.id=543daa9b358b4460a9214ed2004c0ece&scope.project.id=dee508628d3e4a5da2765185fe58ed11&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:37.701502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e961093-6642-4e94-a0aa-1811710ad5ea None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:37.702000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e961093-6642-4e94-a0aa-1811710ad5ea None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:37.702106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e961093-6642-4e94-a0aa-1811710ad5ea None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:37.703352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 568/1147] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:37.708358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c54be-5c76-499b-95ae-f614c694e581 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:37.708732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c54be-5c76-499b-95ae-f614c694e581 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:37.708939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c54be-5c76-499b-95ae-f614c694e581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:37.727729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ca6c54be-5c76-499b-95ae-f614c694e581 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:37.728932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ca6c54be-5c76-499b-95ae-f614c694e581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:37.793867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 580/1148] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:38.061290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e961093-6642-4e94-a0aa-1811710ad5ea None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:38.131599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:38.161528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e961093-6642-4e94-a0aa-1811710ad5ea None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:38.195420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c549a-147c-419b-9ccf-193ef9f18faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:38.196026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c549a-147c-419b-9ccf-193ef9f18faf None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:38.196299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c549a-147c-419b-9ccf-193ef9f18faf None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:38.196567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c549a-147c-419b-9ccf-193ef9f18faf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:38.206378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c549a-147c-419b-9ccf-193ef9f18faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-598c549a-147c-419b-9ccf-193ef9f18faf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:38.211525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:38.212587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:38.213640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 569/1149] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:13:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:38.218595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c54be-5c76-499b-95ae-f614c694e581 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:38.262991 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:38.313310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c54be-5c76-499b-95ae-f614c694e581 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:38.346237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76390540-008d-4c1e-a7d4-a4ef9bafc617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:38.346823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76390540-008d-4c1e-a7d4-a4ef9bafc617 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:38.347098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76390540-008d-4c1e-a7d4-a4ef9bafc617 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:38.347364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76390540-008d-4c1e-a7d4-a4ef9bafc617 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:38.348004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76390540-008d-4c1e-a7d4-a4ef9bafc617 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:38.348297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76390540-008d-4c1e-a7d4-a4ef9bafc617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:38.349703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76390540-008d-4c1e-a7d4-a4ef9bafc617 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:38.357389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 581/1150] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:13:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:38.363471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c549a-147c-419b-9ccf-193ef9f18faf None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:38.460524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c549a-147c-419b-9ccf-193ef9f18faf None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:38.487350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:38.493594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63544675-6dd2-4933-be15-cca8baee3626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:38.494242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63544675-6dd2-4933-be15-cca8baee3626 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:38.494515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63544675-6dd2-4933-be15-cca8baee3626 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:38.494782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63544675-6dd2-4933-be15-cca8baee3626 None admin] PATH_INFO: `/v3/users/543daa9b358b4460a9214ed2004c0ece` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:38.495916 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63544675-6dd2-4933-be15-cca8baee3626 None admin] RBAC: Authorizing `identity:get_user(user_id=543daa9b358b4460a9214ed2004c0ece)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:38.497362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63544675-6dd2-4933-be15-cca8baee3626 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:38.499073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 570/1151] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:38 2022] GET /identity/v3/users/543daa9b358b4460a9214ed2004c0ece => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:38.504837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76390540-008d-4c1e-a7d4-a4ef9bafc617 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:38.600640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76390540-008d-4c1e-a7d4-a4ef9bafc617 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:38.609288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:38.633791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24792c3f-4c2b-4a26-ab65-7ec29e0783af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:38.634402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24792c3f-4c2b-4a26-ab65-7ec29e0783af None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:38.634677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24792c3f-4c2b-4a26-ab65-7ec29e0783af None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:38.634946 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24792c3f-4c2b-4a26-ab65-7ec29e0783af None admin] PATH_INFO: `/v3/projects/dee508628d3e4a5da2765185fe58ed11` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:38.636041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24792c3f-4c2b-4a26-ab65-7ec29e0783af None admin] RBAC: Authorizing `identity:get_project(project_id=dee508628d3e4a5da2765185fe58ed11)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:38.637488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24792c3f-4c2b-4a26-ab65-7ec29e0783af None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:38.639102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 582/1152] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:38 2022] GET /identity/v3/projects/dee508628d3e4a5da2765185fe58ed11 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:38.645001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63544675-6dd2-4933-be15-cca8baee3626 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:38.742089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63544675-6dd2-4933-be15-cca8baee3626 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:38.775481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0e33f-23a8-4303-97c6-e452b9571014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:38.776152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e33f-23a8-4303-97c6-e452b9571014 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:38.776431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e33f-23a8-4303-97c6-e452b9571014 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:38.776729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e33f-23a8-4303-97c6-e452b9571014 None admin] PATH_INFO: `/v3/projects/dee508628d3e4a5da2765185fe58ed11/users/543daa9b358b4460a9214ed2004c0ece/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:38.778630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e33f-23a8-4303-97c6-e452b9571014 None admin] RBAC: Authorizing `identity:create_grant(project_id=dee508628d3e4a5da2765185fe58ed11, user_id=543daa9b358b4460a9214ed2004c0ece, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:38.780081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e33f-23a8-4303-97c6-e452b9571014 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:38.888133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 571/1153] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:13:38 2022] PUT /identity/v3/projects/dee508628d3e4a5da2765185fe58ed11/users/543daa9b358b4460a9214ed2004c0ece/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:13:40.774457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c963f-470b-4c0e-953a-29c66bbb4cff None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:40.775000 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c963f-470b-4c0e-953a-29c66bbb4cff None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:40.775000 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c963f-470b-4c0e-953a-29c66bbb4cff None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:40.776355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 583/1154] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:40.781734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e67001-f9d5-476c-838e-f7bea473c90c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:40.782130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e67001-f9d5-476c-838e-f7bea473c90c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:40.782272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e67001-f9d5-476c-838e-f7bea473c90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:40.800469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c7e67001-f9d5-476c-838e-f7bea473c90c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:40.801656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c7e67001-f9d5-476c-838e-f7bea473c90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:40.853844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:40.867418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 572/1155] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:41.139727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c963f-470b-4c0e-953a-29c66bbb4cff None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:41.241744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c963f-470b-4c0e-953a-29c66bbb4cff None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:41.283456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd9720-0ef2-48f7-a59a-072c0e7d627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:41.284072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd9720-0ef2-48f7-a59a-072c0e7d627a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:41.284348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd9720-0ef2-48f7-a59a-072c0e7d627a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:41.284614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd9720-0ef2-48f7-a59a-072c0e7d627a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:41.295031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd9720-0ef2-48f7-a59a-072c0e7d627a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8efd9720-0ef2-48f7-a59a-072c0e7d627a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:41.299643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:41.300734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:41.302041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 584/1156] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:13:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:41.307142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e67001-f9d5-476c-838e-f7bea473c90c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:41.408118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e67001-f9d5-476c-838e-f7bea473c90c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:41.441354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452df16-65db-4fab-b614-5873351c0972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:41.441942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452df16-65db-4fab-b614-5873351c0972 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:41.442216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452df16-65db-4fab-b614-5873351c0972 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:41.442480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452df16-65db-4fab-b614-5873351c0972 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:41.443122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452df16-65db-4fab-b614-5873351c0972 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:41.444073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452df16-65db-4fab-b614-5873351c0972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:41.445431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452df16-65db-4fab-b614-5873351c0972 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:41.453340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 573/1157] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:41.459228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd9720-0ef2-48f7-a59a-072c0e7d627a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:41.494945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:41.557993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd9720-0ef2-48f7-a59a-072c0e7d627a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:41.590308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185f1f7-defe-4928-9a52-8a3642c917eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:41.590919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185f1f7-defe-4928-9a52-8a3642c917eb None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:41.591191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185f1f7-defe-4928-9a52-8a3642c917eb None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:41.591457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185f1f7-defe-4928-9a52-8a3642c917eb None admin] PATH_INFO: `/v3/users/543daa9b358b4460a9214ed2004c0ece` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:41.592559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185f1f7-defe-4928-9a52-8a3642c917eb None admin] RBAC: Authorizing `identity:get_user(user_id=543daa9b358b4460a9214ed2004c0ece)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:41.594013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185f1f7-defe-4928-9a52-8a3642c917eb None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:41.595586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 585/1158] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:41 2022] GET /identity/v3/users/543daa9b358b4460a9214ed2004c0ece => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:41.602242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452df16-65db-4fab-b614-5873351c0972 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:41.651859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:41.701282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452df16-65db-4fab-b614-5873351c0972 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:41.735219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4cb57-d411-4ac6-8b5d-b00d67f5e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:41.735810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4cb57-d411-4ac6-8b5d-b00d67f5e41f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:41.736084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4cb57-d411-4ac6-8b5d-b00d67f5e41f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:41.736351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4cb57-d411-4ac6-8b5d-b00d67f5e41f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:41.801645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4cb57-d411-4ac6-8b5d-b00d67f5e41f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:41.803290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4cb57-d411-4ac6-8b5d-b00d67f5e41f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:41.850724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:41.900247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 574/1159] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:13:41.906196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185f1f7-defe-4928-9a52-8a3642c917eb None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:42.004834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185f1f7-defe-4928-9a52-8a3642c917eb None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:42.007793 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:42.038680 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f2aac-7aa0-4986-b56f-7e9e093221cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:42.039295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f2aac-7aa0-4986-b56f-7e9e093221cd None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:42.039573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f2aac-7aa0-4986-b56f-7e9e093221cd None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:42.039847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f2aac-7aa0-4986-b56f-7e9e093221cd None admin] PATH_INFO: `/v3/projects/dee508628d3e4a5da2765185fe58ed11` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:42.040983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f2aac-7aa0-4986-b56f-7e9e093221cd None admin] RBAC: Authorizing `identity:get_project(project_id=dee508628d3e4a5da2765185fe58ed11)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:42.042446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f2aac-7aa0-4986-b56f-7e9e093221cd None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:42.044062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 586/1160] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:41 2022] GET /identity/v3/projects/dee508628d3e4a5da2765185fe58ed11 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:42.050166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4cb57-d411-4ac6-8b5d-b00d67f5e41f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:42.146961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4cb57-d411-4ac6-8b5d-b00d67f5e41f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:42.180120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c55736-edcd-4e5d-8350-9d85b5b0c540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:42.180704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c55736-edcd-4e5d-8350-9d85b5b0c540 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:42.181005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c55736-edcd-4e5d-8350-9d85b5b0c540 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:42.181277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c55736-edcd-4e5d-8350-9d85b5b0c540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:42.181981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c55736-edcd-4e5d-8350-9d85b5b0c540 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=dee508628d3e4a5da2765185fe58ed11, user.id=543daa9b358b4460a9214ed2004c0ece) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:42.182284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c55736-edcd-4e5d-8350-9d85b5b0c540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:42.183684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c55736-edcd-4e5d-8350-9d85b5b0c540 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:42.189255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 575/1161] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:13:42 2022] GET /identity/v3/role_assignments?user.id=543daa9b358b4460a9214ed2004c0ece&scope.project.id=dee508628d3e4a5da2765185fe58ed11&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:44.106937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772b29-00ab-437c-b1ef-f13b594a966c None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:44.107516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772b29-00ab-437c-b1ef-f13b594a966c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:44.107516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be772b29-00ab-437c-b1ef-f13b594a966c None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:44.108917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 587/1162] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:44.114277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8659bec-874a-4e3a-93f9-f50294498e3f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:44.114656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8659bec-874a-4e3a-93f9-f50294498e3f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:44.114820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8659bec-874a-4e3a-93f9-f50294498e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:44.133358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f8659bec-874a-4e3a-93f9-f50294498e3f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:44.134531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f8659bec-874a-4e3a-93f9-f50294498e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:44.143562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:44.200191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 576/1163] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:44.467335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be772b29-00ab-437c-b1ef-f13b594a966c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:44.565419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be772b29-00ab-437c-b1ef-f13b594a966c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:44.597534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45748b-d015-4278-ad50-99ea5aa3c4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:44.598120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45748b-d015-4278-ad50-99ea5aa3c4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:44.598396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45748b-d015-4278-ad50-99ea5aa3c4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:44.598663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45748b-d015-4278-ad50-99ea5aa3c4e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:44.638032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:44.664770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45748b-d015-4278-ad50-99ea5aa3c4e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:44.666239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45748b-d015-4278-ad50-99ea5aa3c4e7 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:44.761297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 588/1164] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:13:44.767076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8659bec-874a-4e3a-93f9-f50294498e3f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:44.864050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8659bec-874a-4e3a-93f9-f50294498e3f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:44.897933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b0e46-660b-48b4-a11a-a68bd2232692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:44.898544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b0e46-660b-48b4-a11a-a68bd2232692 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:44.898818 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b0e46-660b-48b4-a11a-a68bd2232692 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:44.899089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b0e46-660b-48b4-a11a-a68bd2232692 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:44.905603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b0e46-660b-48b4-a11a-a68bd2232692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:44.907018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b0e46-660b-48b4-a11a-a68bd2232692 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-513b0e46-660b-48b4-a11a-a68bd2232692 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:44.917942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:44.919154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:13:44.920264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 577/1165] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:13:44 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:44.925076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45748b-d015-4278-ad50-99ea5aa3c4e7 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:44.996365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:45.025230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45748b-d015-4278-ad50-99ea5aa3c4e7 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:45.057826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb7b93-1aa8-443e-b2ea-21cd11bd42c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:45.058412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb7b93-1aa8-443e-b2ea-21cd11bd42c9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:45.058688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb7b93-1aa8-443e-b2ea-21cd11bd42c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:45.058956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb7b93-1aa8-443e-b2ea-21cd11bd42c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:45.059579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb7b93-1aa8-443e-b2ea-21cd11bd42c9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:45.059873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb7b93-1aa8-443e-b2ea-21cd11bd42c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:45.061276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb7b93-1aa8-443e-b2ea-21cd11bd42c9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:45.068092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 589/1166] 10.222.0.84 () {56 vars in 1115 bytes} [Wed Aug 24 11:13:44 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:46.989372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b6351-e34e-49d8-9264-906cc2867b36 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:46.989972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b6351-e34e-49d8-9264-906cc2867b36 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:46.989972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b6351-e34e-49d8-9264-906cc2867b36 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:46.991966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 578/1167] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:46.997255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430238d7-572a-4f2c-a6c4-b52a7f077ea2 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:46.997671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430238d7-572a-4f2c-a6c4-b52a7f077ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:46.997779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430238d7-572a-4f2c-a6c4-b52a7f077ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:47.016094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-430238d7-572a-4f2c-a6c4-b52a7f077ea2 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:47.017313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-430238d7-572a-4f2c-a6c4-b52a7f077ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:47.082007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 590/1168] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:47.353669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b6351-e34e-49d8-9264-906cc2867b36 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:47.357522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:47.453981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b6351-e34e-49d8-9264-906cc2867b36 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:47.486286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211dd854-754a-4adb-868f-c6752d686953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:47.486868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dd854-754a-4adb-868f-c6752d686953 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:47.487142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dd854-754a-4adb-868f-c6752d686953 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:47.487409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dd854-754a-4adb-868f-c6752d686953 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:47.488050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211dd854-754a-4adb-868f-c6752d686953 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:47.489423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211dd854-754a-4adb-868f-c6752d686953 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:47.569430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 579/1169] 10.222.0.84 () {60 vars in 1138 bytes} [Wed Aug 24 11:13:47 2022] POST /identity/v3/domains => generated 267 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:13:49.493775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36248b-b1b1-4d67-bcf8-7b41f9cfc7c0 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:49.494309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36248b-b1b1-4d67-bcf8-7b41f9cfc7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:49.494309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36248b-b1b1-4d67-bcf8-7b41f9cfc7c0 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:49.495654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 591/1170] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:49.500999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517ba9c-22d3-4a17-ba81-f8964fbb2ec9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:49.501392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517ba9c-22d3-4a17-ba81-f8964fbb2ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:49.501554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517ba9c-22d3-4a17-ba81-f8964fbb2ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:49.519661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f517ba9c-22d3-4a17-ba81-f8964fbb2ec9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:49.520871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f517ba9c-22d3-4a17-ba81-f8964fbb2ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:49.580084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:49.588749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 580/1171] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:49.854788 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36248b-b1b1-4d67-bcf8-7b41f9cfc7c0 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:49.919857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:49.952609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36248b-b1b1-4d67-bcf8-7b41f9cfc7c0 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:49.986240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb3491-504c-49c2-be87-7fa15ddad3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:49.986851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb3491-504c-49c2-be87-7fa15ddad3bf None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:49.987125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb3491-504c-49c2-be87-7fa15ddad3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:49.987391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb3491-504c-49c2-be87-7fa15ddad3bf None admin] PATH_INFO: `/v3/domains/8588a7e4bf0345f3b319748746b2ff37` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:49.997310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb3491-504c-49c2-be87-7fa15ddad3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=8588a7e4bf0345f3b319748746b2ff37)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:49.998728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb3491-504c-49c2-be87-7fa15ddad3bf None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:50.000298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 592/1172] 10.222.0.84 () {56 vars in 1183 bytes} [Wed Aug 24 11:13:49 2022] GET /identity/v3/domains/8588a7e4bf0345f3b319748746b2ff37 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:50.006754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517ba9c-22d3-4a17-ba81-f8964fbb2ec9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:50.102250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517ba9c-22d3-4a17-ba81-f8964fbb2ec9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:50.135340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a95c1f-6fed-4923-9423-81f3b6953070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:50.135867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a95c1f-6fed-4923-9423-81f3b6953070 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:50.136145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a95c1f-6fed-4923-9423-81f3b6953070 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:50.136417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a95c1f-6fed-4923-9423-81f3b6953070 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:50.137082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a95c1f-6fed-4923-9423-81f3b6953070 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:50.138500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a95c1f-6fed-4923-9423-81f3b6953070 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:50.243878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 581/1173] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:13:50 2022] POST /identity/v3/projects => generated 361 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:13:52.166396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46b4d4-10f4-4854-abfc-404c619ea070 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:52.166987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46b4d4-10f4-4854-abfc-404c619ea070 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:52.166987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c46b4d4-10f4-4854-abfc-404c619ea070 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:52.168268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 593/1174] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:52.173365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cbc31-fe88-4147-8f83-5aa30ac4b554 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:52.173787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cbc31-fe88-4147-8f83-5aa30ac4b554 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:52.173916 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cbc31-fe88-4147-8f83-5aa30ac4b554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:52.192018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0d4cbc31-fe88-4147-8f83-5aa30ac4b554 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:52.193843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0d4cbc31-fe88-4147-8f83-5aa30ac4b554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:52.234554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:52.259243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 582/1175] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:52.528836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46b4d4-10f4-4854-abfc-404c619ea070 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:52.626140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c46b4d4-10f4-4854-abfc-404c619ea070 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:52.653055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:52.661635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfdf0d-231f-4d2a-a752-d7aaaf8a2134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:52.663511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfdf0d-231f-4d2a-a752-d7aaaf8a2134 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:52.663792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfdf0d-231f-4d2a-a752-d7aaaf8a2134 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:52.664061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfdf0d-231f-4d2a-a752-d7aaaf8a2134 None admin] PATH_INFO: `/v3/domains/8588a7e4bf0345f3b319748746b2ff37` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:52.665212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfdf0d-231f-4d2a-a752-d7aaaf8a2134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8588a7e4bf0345f3b319748746b2ff37)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:52.666634 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfdf0d-231f-4d2a-a752-d7aaaf8a2134 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:52.668206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 594/1176] 10.222.0.84 () {56 vars in 1183 bytes} [Wed Aug 24 11:13:52 2022] GET /identity/v3/domains/8588a7e4bf0345f3b319748746b2ff37 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:52.673891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cbc31-fe88-4147-8f83-5aa30ac4b554 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:52.770700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cbc31-fe88-4147-8f83-5aa30ac4b554 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:52.804695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226713d-17d7-40d5-bf32-c2009d8cae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:52.805262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226713d-17d7-40d5-bf32-c2009d8cae34 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:52.805538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226713d-17d7-40d5-bf32-c2009d8cae34 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:52.805808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226713d-17d7-40d5-bf32-c2009d8cae34 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:52.806442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226713d-17d7-40d5-bf32-c2009d8cae34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:52.807856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226713d-17d7-40d5-bf32-c2009d8cae34 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:52.900025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 583/1177] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:13:52 2022] POST /identity/v3/users => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:13:54.831923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89496f-9c82-4a22-96dc-a93a36e290d9 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:54.832465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89496f-9c82-4a22-96dc-a93a36e290d9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:54.832465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89496f-9c82-4a22-96dc-a93a36e290d9 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:54.834389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 595/1178] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:54.839620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b97bb-1b68-43dd-aab0-121026336008 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:54.840042 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b97bb-1b68-43dd-aab0-121026336008 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:54.840167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b97bb-1b68-43dd-aab0-121026336008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:54.858019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d93b97bb-1b68-43dd-aab0-121026336008 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:54.859193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d93b97bb-1b68-43dd-aab0-121026336008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:54.885987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:54.925689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 584/1179] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:55.196659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89496f-9c82-4a22-96dc-a93a36e290d9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:55.295060 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89496f-9c82-4a22-96dc-a93a36e290d9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:55.327976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54912890-4308-42e2-969a-217f2ebf1542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:55.328576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54912890-4308-42e2-969a-217f2ebf1542 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:55.328881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54912890-4308-42e2-969a-217f2ebf1542 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:55.329147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54912890-4308-42e2-969a-217f2ebf1542 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:55.339251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54912890-4308-42e2-969a-217f2ebf1542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-54912890-4308-42e2-969a-217f2ebf1542 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:55.343829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:55.345015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:55.346102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 596/1180] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:13:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:55.351478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b97bb-1b68-43dd-aab0-121026336008 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:55.453561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b97bb-1b68-43dd-aab0-121026336008 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:55.487001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07398eb-5aab-46c7-a058-8acd0041d2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:55.487596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07398eb-5aab-46c7-a058-8acd0041d2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:55.487900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07398eb-5aab-46c7-a058-8acd0041d2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:55.488165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07398eb-5aab-46c7-a058-8acd0041d2f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:55.488830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07398eb-5aab-46c7-a058-8acd0041d2f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:55.489131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07398eb-5aab-46c7-a058-8acd0041d2f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:55.491116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07398eb-5aab-46c7-a058-8acd0041d2f1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:55.499173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 585/1181] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:13:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:55.505156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54912890-4308-42e2-969a-217f2ebf1542 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:55.559527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:55.602379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54912890-4308-42e2-969a-217f2ebf1542 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:55.635008 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fb215-6630-4b19-b7a2-c1f85e1de064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:55.635621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fb215-6630-4b19-b7a2-c1f85e1de064 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:55.635895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fb215-6630-4b19-b7a2-c1f85e1de064 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:55.636163 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fb215-6630-4b19-b7a2-c1f85e1de064 None admin] PATH_INFO: `/v3/users/ba476f940d27406180f66a2e9210a3e5` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:55.649341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fb215-6630-4b19-b7a2-c1f85e1de064 None admin] RBAC: Authorizing `identity:get_user(user_id=ba476f940d27406180f66a2e9210a3e5)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:55.650770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fb215-6630-4b19-b7a2-c1f85e1de064 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:55.652393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 597/1182] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:55 2022] GET /identity/v3/users/ba476f940d27406180f66a2e9210a3e5 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:13:55.658293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07398eb-5aab-46c7-a058-8acd0041d2f1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:55.684203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:55.757836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07398eb-5aab-46c7-a058-8acd0041d2f1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:55.790823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09dac9a-e289-4fa1-ac0e-b30e03986302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:55.791405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dac9a-e289-4fa1-ac0e-b30e03986302 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:55.791679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dac9a-e289-4fa1-ac0e-b30e03986302 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:55.791945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09dac9a-e289-4fa1-ac0e-b30e03986302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:55.857281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09dac9a-e289-4fa1-ac0e-b30e03986302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:55.858711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09dac9a-e289-4fa1-ac0e-b30e03986302 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:55.881444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:55.953005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 586/1183] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:13:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:13:55.959541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fb215-6630-4b19-b7a2-c1f85e1de064 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:56.055661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fb215-6630-4b19-b7a2-c1f85e1de064 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:56.069433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:56.089670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595ef1e-f5d2-464c-a76d-7e69112b3f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:56.092333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ef1e-f5d2-464c-a76d-7e69112b3f5e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:56.092333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ef1e-f5d2-464c-a76d-7e69112b3f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:56.092333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ef1e-f5d2-464c-a76d-7e69112b3f5e None admin] PATH_INFO: `/v3/projects/8428d8bc41cc486e97e1a8310572d751` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:56.092333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595ef1e-f5d2-464c-a76d-7e69112b3f5e None admin] RBAC: Authorizing `identity:get_project(project_id=8428d8bc41cc486e97e1a8310572d751)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:56.093283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595ef1e-f5d2-464c-a76d-7e69112b3f5e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:56.094884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 598/1184] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:55 2022] GET /identity/v3/projects/8428d8bc41cc486e97e1a8310572d751 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:56.101185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09dac9a-e289-4fa1-ac0e-b30e03986302 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:56.195309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09dac9a-e289-4fa1-ac0e-b30e03986302 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:56.217832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:56.227943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216e418-6dc9-4b84-8f75-aa0edec4460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:56.228521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216e418-6dc9-4b84-8f75-aa0edec4460d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:56.228818 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216e418-6dc9-4b84-8f75-aa0edec4460d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:56.229091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216e418-6dc9-4b84-8f75-aa0edec4460d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:56.229790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216e418-6dc9-4b84-8f75-aa0edec4460d None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=8428d8bc41cc486e97e1a8310572d751, user.id=ba476f940d27406180f66a2e9210a3e5) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:56.230087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216e418-6dc9-4b84-8f75-aa0edec4460d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:56.231483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216e418-6dc9-4b84-8f75-aa0edec4460d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:56.236897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 587/1185] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:13:56 2022] GET /identity/v3/role_assignments?user.id=ba476f940d27406180f66a2e9210a3e5&scope.project.id=8428d8bc41cc486e97e1a8310572d751&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:58.135136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a000324-6169-441b-895b-4252c322536a None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:58.135658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a000324-6169-441b-895b-4252c322536a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:58.135658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a000324-6169-441b-895b-4252c322536a None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:58.137071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 599/1186] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:13:58.142241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd4ed4-7d03-4b75-a78a-fa7ebee95400 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:58.142660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd4ed4-7d03-4b75-a78a-fa7ebee95400 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:58.142789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd4ed4-7d03-4b75-a78a-fa7ebee95400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:58.160320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-47bd4ed4-7d03-4b75-a78a-fa7ebee95400 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:13:58.161533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-47bd4ed4-7d03-4b75-a78a-fa7ebee95400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:13:58.226044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 588/1187] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:13:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:13:58.501122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a000324-6169-441b-895b-4252c322536a None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:58.597579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a000324-6169-441b-895b-4252c322536a None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:58.630358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd718169-a705-451a-a4bb-5f990d9d5d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:58.630969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd718169-a705-451a-a4bb-5f990d9d5d79 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:58.631241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd718169-a705-451a-a4bb-5f990d9d5d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:58.631510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd718169-a705-451a-a4bb-5f990d9d5d79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:58.641093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd718169-a705-451a-a4bb-5f990d9d5d79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-dd718169-a705-451a-a4bb-5f990d9d5d79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:13:58.645649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:13:58.647570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:13:58.649207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 600/1188] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:13:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:13:58.656165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd4ed4-7d03-4b75-a78a-fa7ebee95400 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:58.768479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd4ed4-7d03-4b75-a78a-fa7ebee95400 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:58.785674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:58.802455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484acf8a-5628-46f7-a693-735f1b9c2d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:58.807548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484acf8a-5628-46f7-a693-735f1b9c2d91 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:58.807548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484acf8a-5628-46f7-a693-735f1b9c2d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:58.807548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484acf8a-5628-46f7-a693-735f1b9c2d91 None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:58.807548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484acf8a-5628-46f7-a693-735f1b9c2d91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:13:58.807548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484acf8a-5628-46f7-a693-735f1b9c2d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:58.807548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484acf8a-5628-46f7-a693-735f1b9c2d91 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:58.813588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 589/1189] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:13:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:58.819580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd718169-a705-451a-a4bb-5f990d9d5d79 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:58.861486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:58.917448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd718169-a705-451a-a4bb-5f990d9d5d79 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:58.949100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00511a-0ab1-4839-ad0f-b440aff41428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:58.949708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00511a-0ab1-4839-ad0f-b440aff41428 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:58.949980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00511a-0ab1-4839-ad0f-b440aff41428 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:58.950245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00511a-0ab1-4839-ad0f-b440aff41428 None admin] PATH_INFO: `/v3/users/ba476f940d27406180f66a2e9210a3e5` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:58.951332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00511a-0ab1-4839-ad0f-b440aff41428 None admin] RBAC: Authorizing `identity:get_user(user_id=ba476f940d27406180f66a2e9210a3e5)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:58.952738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00511a-0ab1-4839-ad0f-b440aff41428 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:58.954311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 601/1190] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:13:58 2022] GET /identity/v3/users/ba476f940d27406180f66a2e9210a3e5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:58.960570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484acf8a-5628-46f7-a693-735f1b9c2d91 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:59.060095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484acf8a-5628-46f7-a693-735f1b9c2d91 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:59.093222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67669432-8dfc-4f91-87c8-3a07bfaee025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:59.093953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669432-8dfc-4f91-87c8-3a07bfaee025 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:59.094324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669432-8dfc-4f91-87c8-3a07bfaee025 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:59.094687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67669432-8dfc-4f91-87c8-3a07bfaee025 None admin] PATH_INFO: `/v3/projects/8428d8bc41cc486e97e1a8310572d751` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:59.095919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67669432-8dfc-4f91-87c8-3a07bfaee025 None admin] RBAC: Authorizing `identity:get_project(project_id=8428d8bc41cc486e97e1a8310572d751)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:59.097475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67669432-8dfc-4f91-87c8-3a07bfaee025 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:59.099234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 590/1191] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:13:58 2022] GET /identity/v3/projects/8428d8bc41cc486e97e1a8310572d751 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:13:59.105398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00511a-0ab1-4839-ad0f-b440aff41428 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:13:59.201610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00511a-0ab1-4839-ad0f-b440aff41428 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:13:59.210937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:13:59.234084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2876e7b-51c8-4fad-88ca-b76b7294cebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:13:59.234973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2876e7b-51c8-4fad-88ca-b76b7294cebd None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:13:59.235301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2876e7b-51c8-4fad-88ca-b76b7294cebd None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:13:59.235524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2876e7b-51c8-4fad-88ca-b76b7294cebd None admin] PATH_INFO: `/v3/projects/8428d8bc41cc486e97e1a8310572d751/users/ba476f940d27406180f66a2e9210a3e5/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:13:59.237892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2876e7b-51c8-4fad-88ca-b76b7294cebd None admin] RBAC: Authorizing `identity:create_grant(project_id=8428d8bc41cc486e97e1a8310572d751, user_id=ba476f940d27406180f66a2e9210a3e5, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:13:59.239311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2876e7b-51c8-4fad-88ca-b76b7294cebd None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:13:59.316460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 602/1192] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:13:59 2022] PUT /identity/v3/projects/8428d8bc41cc486e97e1a8310572d751/users/ba476f940d27406180f66a2e9210a3e5/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:14:01.238010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b80c8d-545f-4402-b1eb-32cfa7fa66d7 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:01.238562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b80c8d-545f-4402-b1eb-32cfa7fa66d7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:01.238562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b80c8d-545f-4402-b1eb-32cfa7fa66d7 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:01.240535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 591/1193] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:14:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:14:01.246172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54b1ae-3c67-4a97-bf22-e7f4bff9e207 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:01.246588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54b1ae-3c67-4a97-bf22-e7f4bff9e207 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:01.246707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54b1ae-3c67-4a97-bf22-e7f4bff9e207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:01.264806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ef54b1ae-3c67-4a97-bf22-e7f4bff9e207 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:14:01.265990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ef54b1ae-3c67-4a97-bf22-e7f4bff9e207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:14:01.328624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 603/1194] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:14:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:14:01.596748 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b80c8d-545f-4402-b1eb-32cfa7fa66d7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:01.648979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:01.698550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b80c8d-545f-4402-b1eb-32cfa7fa66d7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:01.731805 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bacd3b-b7bf-490e-b9f7-f66da8dee2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:01.732412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bacd3b-b7bf-490e-b9f7-f66da8dee2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:01.732683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bacd3b-b7bf-490e-b9f7-f66da8dee2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:01.732979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bacd3b-b7bf-490e-b9f7-f66da8dee2c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:01.743311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bacd3b-b7bf-490e-b9f7-f66da8dee2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-64bacd3b-b7bf-490e-b9f7-f66da8dee2c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:14:01.747849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:14:01.748991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:14:01.750057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:14:01.752945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 592/1195] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:14:01.754896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54b1ae-3c67-4a97-bf22-e7f4bff9e207 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:01.851604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:01.854992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54b1ae-3c67-4a97-bf22-e7f4bff9e207 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:01.887351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707c829-96f1-4275-b1aa-338bf93c18e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:01.887963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707c829-96f1-4275-b1aa-338bf93c18e5 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:01.888219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707c829-96f1-4275-b1aa-338bf93c18e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:01.888482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707c829-96f1-4275-b1aa-338bf93c18e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:01.889151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707c829-96f1-4275-b1aa-338bf93c18e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:14:01.893354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707c829-96f1-4275-b1aa-338bf93c18e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:14:01.893354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707c829-96f1-4275-b1aa-338bf93c18e5 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:14:01.899101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 604/1196] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:14:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:14:01.904881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bacd3b-b7bf-490e-b9f7-f66da8dee2c2 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:02.002850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bacd3b-b7bf-490e-b9f7-f66da8dee2c2 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:02.012309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:02.036822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e8c7-57cb-4dd0-a8a5-a1e01d27f040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:02.037523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e8c7-57cb-4dd0-a8a5-a1e01d27f040 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:02.037799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e8c7-57cb-4dd0-a8a5-a1e01d27f040 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:02.038126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e8c7-57cb-4dd0-a8a5-a1e01d27f040 None admin] PATH_INFO: `/v3/users/ba476f940d27406180f66a2e9210a3e5` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:02.039363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e8c7-57cb-4dd0-a8a5-a1e01d27f040 None admin] RBAC: Authorizing `identity:get_user(user_id=ba476f940d27406180f66a2e9210a3e5)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:14:02.040772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6e8c7-57cb-4dd0-a8a5-a1e01d27f040 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:14:02.042648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 593/1197] 10.222.0.84 () {56 vars in 1177 bytes} [Wed Aug 24 11:14:01 2022] GET /identity/v3/users/ba476f940d27406180f66a2e9210a3e5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:14:02.048319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707c829-96f1-4275-b1aa-338bf93c18e5 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:02.156123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707c829-96f1-4275-b1aa-338bf93c18e5 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:02.189052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2d30b-8cd2-4ce9-b309-361541e9aaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:02.189663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2d30b-8cd2-4ce9-b309-361541e9aaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:02.189944 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2d30b-8cd2-4ce9-b309-361541e9aaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:02.190223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2d30b-8cd2-4ce9-b309-361541e9aaa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:02.217588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:02.255727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2d30b-8cd2-4ce9-b309-361541e9aaa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:14:02.257193 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2d30b-8cd2-4ce9-b309-361541e9aaa6 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:14:02.350392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 605/1198] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:14:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:14:02.356398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e8c7-57cb-4dd0-a8a5-a1e01d27f040 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:02.458631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e8c7-57cb-4dd0-a8a5-a1e01d27f040 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:02.493979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46360334-f693-4492-b780-c85e70d7688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:02.494591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46360334-f693-4492-b780-c85e70d7688d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:02.494863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46360334-f693-4492-b780-c85e70d7688d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:02.495131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46360334-f693-4492-b780-c85e70d7688d None admin] PATH_INFO: `/v3/projects/8428d8bc41cc486e97e1a8310572d751` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:02.496219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46360334-f693-4492-b780-c85e70d7688d None admin] RBAC: Authorizing `identity:get_project(project_id=8428d8bc41cc486e97e1a8310572d751)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:14:02.497655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46360334-f693-4492-b780-c85e70d7688d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:14:02.499267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 594/1199] 10.222.0.84 () {56 vars in 1186 bytes} [Wed Aug 24 11:14:02 2022] GET /identity/v3/projects/8428d8bc41cc486e97e1a8310572d751 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:14:02.505979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2d30b-8cd2-4ce9-b309-361541e9aaa6 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:14:02.562301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:02.601448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2d30b-8cd2-4ce9-b309-361541e9aaa6 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:14:02.634917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48412a-4d74-478a-9521-c2e895482368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:14:02.635491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48412a-4d74-478a-9521-c2e895482368 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:14:02.635764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48412a-4d74-478a-9521-c2e895482368 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:14:02.636028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48412a-4d74-478a-9521-c2e895482368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:14:02.636729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48412a-4d74-478a-9521-c2e895482368 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=8428d8bc41cc486e97e1a8310572d751, user.id=ba476f940d27406180f66a2e9210a3e5) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:14:02.637054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48412a-4d74-478a-9521-c2e895482368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:14:02.638450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48412a-4d74-478a-9521-c2e895482368 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:14:02.643983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 606/1200] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:14:02 2022] GET /identity/v3/role_assignments?user.id=ba476f940d27406180f66a2e9210a3e5&scope.project.id=8428d8bc41cc486e97e1a8310572d751&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:14:12.489277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=78 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:14:12.641394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=38 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:15.214321 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927623ec-12ca-466d-8e35-8d62de4a3d16 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:15.214902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927623ec-12ca-466d-8e35-8d62de4a3d16 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:15.214902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927623ec-12ca-466d-8e35-8d62de4a3d16 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:15.216197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 595/1201] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:15.221360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36acca4-9344-4f57-bea9-d516710b93f6 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:15.221751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36acca4-9344-4f57-bea9-d516710b93f6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:15.221896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36acca4-9344-4f57-bea9-d516710b93f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:15.239905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a36acca4-9344-4f57-bea9-d516710b93f6 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:15.241190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a36acca4-9344-4f57-bea9-d516710b93f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:15.306525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 607/1202] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:15.576822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927623ec-12ca-466d-8e35-8d62de4a3d16 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:15.678314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927623ec-12ca-466d-8e35-8d62de4a3d16 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:15.711770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cb07e-c36b-437e-9fa7-97d48871e229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:15.712380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cb07e-c36b-437e-9fa7-97d48871e229 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:15.712652 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cb07e-c36b-437e-9fa7-97d48871e229 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:15.712947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cb07e-c36b-437e-9fa7-97d48871e229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:15.722926 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cb07e-c36b-437e-9fa7-97d48871e229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:15.724341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624cb07e-c36b-437e-9fa7-97d48871e229 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-624cb07e-c36b-437e-9fa7-97d48871e229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:15.738542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:15.739763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:15.747481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:15.747481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:15.747481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:15.747481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 596/1203] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:15.747797 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36acca4-9344-4f57-bea9-d516710b93f6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:15.842476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36acca4-9344-4f57-bea9-d516710b93f6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:15.871175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:15.876471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c4ba5-8922-4842-adb0-323895626974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:15.878740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c4ba5-8922-4842-adb0-323895626974 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:15.878740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c4ba5-8922-4842-adb0-323895626974 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:15.878740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c4ba5-8922-4842-adb0-323895626974 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:15.878740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c4ba5-8922-4842-adb0-323895626974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:15.878740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c4ba5-8922-4842-adb0-323895626974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:15.879831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c4ba5-8922-4842-adb0-323895626974 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:15.890858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 608/1204] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:15.896966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cb07e-c36b-437e-9fa7-97d48871e229 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:15.949287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:15.996011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cb07e-c36b-437e-9fa7-97d48871e229 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:16.030318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48506a43-f448-431c-b870-9a33d5d6cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:16.030958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48506a43-f448-431c-b870-9a33d5d6cd31 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:16.031263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48506a43-f448-431c-b870-9a33d5d6cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:16.031533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48506a43-f448-431c-b870-9a33d5d6cd31 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:16.032294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48506a43-f448-431c-b870-9a33d5d6cd31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:16.033790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48506a43-f448-431c-b870-9a33d5d6cd31 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:16.117271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 597/1205] 10.222.0.84 () {60 vars in 1132 bytes} [Wed Aug 24 11:15:15 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:18.070459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035830b-ede0-469a-86cd-c188a77801e5 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:18.075509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035830b-ede0-469a-86cd-c188a77801e5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:18.075509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035830b-ede0-469a-86cd-c188a77801e5 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:18.075509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 609/1206] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:18.077688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e0742-87b1-494e-946f-e5b0c797b13a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:18.078075 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e0742-87b1-494e-946f-e5b0c797b13a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:18.078230 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e0742-87b1-494e-946f-e5b0c797b13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:18.097635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-013e0742-87b1-494e-946f-e5b0c797b13a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:18.098914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-013e0742-87b1-494e-946f-e5b0c797b13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:18.167505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 598/1207] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:18.440681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035830b-ede0-469a-86cd-c188a77801e5 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:18.542248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035830b-ede0-469a-86cd-c188a77801e5 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:18.574747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417614-aa8a-4c50-aa27-f5f0ace03055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:18.575404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417614-aa8a-4c50-aa27-f5f0ace03055 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:18.575750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417614-aa8a-4c50-aa27-f5f0ace03055 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:18.576017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417614-aa8a-4c50-aa27-f5f0ace03055 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:18.588003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a417614-aa8a-4c50-aa27-f5f0ace03055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-5a417614-aa8a-4c50-aa27-f5f0ace03055 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:15:18.592835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:18.601035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:18.602096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 610/1208] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:15:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:18.602973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e0742-87b1-494e-946f-e5b0c797b13a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:18.639993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:18.699873 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e0742-87b1-494e-946f-e5b0c797b13a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:18.733868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb27ab-28bc-4d43-810a-12bba6dc34ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:18.734463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb27ab-28bc-4d43-810a-12bba6dc34ec None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:18.734741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb27ab-28bc-4d43-810a-12bba6dc34ec None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:18.735009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb27ab-28bc-4d43-810a-12bba6dc34ec None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:18.735656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb27ab-28bc-4d43-810a-12bba6dc34ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:18.735954 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb27ab-28bc-4d43-810a-12bba6dc34ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:18.737583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb27ab-28bc-4d43-810a-12bba6dc34ec None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:18.745525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 599/1209] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:15:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:18.751347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417614-aa8a-4c50-aa27-f5f0ace03055 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:18.801576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:18.851587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417614-aa8a-4c50-aa27-f5f0ace03055 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:18.886058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8889a7-be7b-4dbf-b1f6-23026500d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:18.886670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8889a7-be7b-4dbf-b1f6-23026500d18a None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:18.886944 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8889a7-be7b-4dbf-b1f6-23026500d18a None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:18.887250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8889a7-be7b-4dbf-b1f6-23026500d18a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:18.898651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8889a7-be7b-4dbf-b1f6-23026500d18a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:18.900068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8889a7-be7b-4dbf-b1f6-23026500d18a None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-2b8889a7-be7b-4dbf-b1f6-23026500d18a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:18.913959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:18.917873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:18.919010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:18.919010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:18.919010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:18.919010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 611/1210] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:18.921539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb27ab-28bc-4d43-810a-12bba6dc34ec None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:19.018506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:19.021128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb27ab-28bc-4d43-810a-12bba6dc34ec None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:19.053709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23126a8d-e5c3-4608-aca7-822a46d8dd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:19.054298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23126a8d-e5c3-4608-aca7-822a46d8dd7e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:19.054573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23126a8d-e5c3-4608-aca7-822a46d8dd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:19.054840 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23126a8d-e5c3-4608-aca7-822a46d8dd7e None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:19.055465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23126a8d-e5c3-4608-aca7-822a46d8dd7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:19.055764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23126a8d-e5c3-4608-aca7-822a46d8dd7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:19.057138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23126a8d-e5c3-4608-aca7-822a46d8dd7e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:19.068239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 600/1211] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:19.074528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8889a7-be7b-4dbf-b1f6-23026500d18a None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:19.173155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8889a7-be7b-4dbf-b1f6-23026500d18a None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:19.174640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:19.206502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8a0f-1c30-4c15-9ca0-f9d01ec8a700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:19.207112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8a0f-1c30-4c15-9ca0-f9d01ec8a700 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:19.207389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8a0f-1c30-4c15-9ca0-f9d01ec8a700 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:19.207657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae8a0f-1c30-4c15-9ca0-f9d01ec8a700 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:19.215418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae8a0f-1c30-4c15-9ca0-f9d01ec8a700 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:19.217189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae8a0f-1c30-4c15-9ca0-f9d01ec8a700 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-88ae8a0f-1c30-4c15-9ca0-f9d01ec8a700 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:15:19.227885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:19.229248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:19.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 612/1212] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:19.234899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23126a8d-e5c3-4608-aca7-822a46d8dd7e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:19.338330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23126a8d-e5c3-4608-aca7-822a46d8dd7e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:19.372664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126b19e-fac2-42fb-a715-babe9819902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:19.373281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126b19e-fac2-42fb-a715-babe9819902c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:19.373556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126b19e-fac2-42fb-a715-babe9819902c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:19.373824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126b19e-fac2-42fb-a715-babe9819902c None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:19.374529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126b19e-fac2-42fb-a715-babe9819902c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:19.374828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126b19e-fac2-42fb-a715-babe9819902c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:19.376286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126b19e-fac2-42fb-a715-babe9819902c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:19.393493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 601/1213] 10.222.0.84 () {56 vars in 1143 bytes} [Wed Aug 24 11:15:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:19.399374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8a0f-1c30-4c15-9ca0-f9d01ec8a700 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:19.500973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae8a0f-1c30-4c15-9ca0-f9d01ec8a700 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:19.534081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd76dd2-3f1a-4cf4-a29d-01de57bfe1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:19.534676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd76dd2-3f1a-4cf4-a29d-01de57bfe1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:19.534953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd76dd2-3f1a-4cf4-a29d-01de57bfe1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:19.535219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd76dd2-3f1a-4cf4-a29d-01de57bfe1f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:19.565043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:19.602684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd76dd2-3f1a-4cf4-a29d-01de57bfe1f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:19.605312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd76dd2-3f1a-4cf4-a29d-01de57bfe1f9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:19.704224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 613/1214] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:15:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:15:19.711397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126b19e-fac2-42fb-a715-babe9819902c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:19.739454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:19.829833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126b19e-fac2-42fb-a715-babe9819902c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:19.866163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fe6ef-5745-431f-8189-a4639e2e5345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:19.866705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fe6ef-5745-431f-8189-a4639e2e5345 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:19.866976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fe6ef-5745-431f-8189-a4639e2e5345 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:19.867239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fe6ef-5745-431f-8189-a4639e2e5345 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:19.876968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7fe6ef-5745-431f-8189-a4639e2e5345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:19.878387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7fe6ef-5745-431f-8189-a4639e2e5345 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-0a7fe6ef-5745-431f-8189-a4639e2e5345 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:19.894022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:19.895215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:19.896508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:19.896508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:19.896508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:19.896508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 602/1215] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:19.900951 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd76dd2-3f1a-4cf4-a29d-01de57bfe1f9 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:19.963232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:20.000316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd76dd2-3f1a-4cf4-a29d-01de57bfe1f9 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:20.032385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f16a9-4a6d-4237-8c8f-6c0225e52cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:20.033000 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f16a9-4a6d-4237-8c8f-6c0225e52cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:20.033276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f16a9-4a6d-4237-8c8f-6c0225e52cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:20.033544 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f16a9-4a6d-4237-8c8f-6c0225e52cd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:20.034167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f16a9-4a6d-4237-8c8f-6c0225e52cd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:20.034465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f16a9-4a6d-4237-8c8f-6c0225e52cd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:20.035876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f16a9-4a6d-4237-8c8f-6c0225e52cd2 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:20.047721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 614/1216] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:20.054071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fe6ef-5745-431f-8189-a4639e2e5345 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:20.130657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:20.163894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fe6ef-5745-431f-8189-a4639e2e5345 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:20.198738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ba3b2-9655-40ec-b3ae-762328007004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:20.199359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ba3b2-9655-40ec-b3ae-762328007004 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:20.199630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ba3b2-9655-40ec-b3ae-762328007004 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:20.199897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ba3b2-9655-40ec-b3ae-762328007004 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:20.207094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ba3b2-9655-40ec-b3ae-762328007004 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:20.208553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ba3b2-9655-40ec-b3ae-762328007004 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-8f4ba3b2-9655-40ec-b3ae-762328007004 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:15:20.221059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:20.222159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:20.223090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:15:20.223090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:15:20.223090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:15:20.223090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:15:20.223090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:15:20.223090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:15:20.223090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:20.223090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:20.223090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 603/1217] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:20.228270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f16a9-4a6d-4237-8c8f-6c0225e52cd2 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:20.331525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f16a9-4a6d-4237-8c8f-6c0225e52cd2 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:20.353349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:20.364586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230f8c2-3df5-4a5d-85a9-d807cf8c250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:20.365210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230f8c2-3df5-4a5d-85a9-d807cf8c250c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:20.365490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230f8c2-3df5-4a5d-85a9-d807cf8c250c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:20.365760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230f8c2-3df5-4a5d-85a9-d807cf8c250c None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:20.366437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230f8c2-3df5-4a5d-85a9-d807cf8c250c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:20.366735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230f8c2-3df5-4a5d-85a9-d807cf8c250c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:20.368145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230f8c2-3df5-4a5d-85a9-d807cf8c250c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:20.379693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 615/1218] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:20.386641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ba3b2-9655-40ec-b3ae-762328007004 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:20.488308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ba3b2-9655-40ec-b3ae-762328007004 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:20.516602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:20.522759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebe811-29ce-42a7-91f9-8e926d5c0fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:20.523351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebe811-29ce-42a7-91f9-8e926d5c0fff None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:20.523625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebe811-29ce-42a7-91f9-8e926d5c0fff None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:20.523893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebe811-29ce-42a7-91f9-8e926d5c0fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:20.524600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebe811-29ce-42a7-91f9-8e926d5c0fff None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=c7cc35f5ddf0446e90ded5c85f21f37d) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:20.524929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebe811-29ce-42a7-91f9-8e926d5c0fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:20.526338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebe811-29ce-42a7-91f9-8e926d5c0fff None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:20.531488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 604/1219] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:15:20 2022] GET /identity/v3/role_assignments?user.id=c7cc35f5ddf0446e90ded5c85f21f37d&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:22.411035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3a37e-cf40-4223-8cf8-fafd9985cb67 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:22.411626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3a37e-cf40-4223-8cf8-fafd9985cb67 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:22.411626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3a37e-cf40-4223-8cf8-fafd9985cb67 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:22.414089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 616/1220] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:22.419060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05159bec-8bc5-4c33-b9e0-8dceb82751b4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:22.419468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05159bec-8bc5-4c33-b9e0-8dceb82751b4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:22.419600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05159bec-8bc5-4c33-b9e0-8dceb82751b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:22.437262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-05159bec-8bc5-4c33-b9e0-8dceb82751b4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:22.438438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-05159bec-8bc5-4c33-b9e0-8dceb82751b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:22.505435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 605/1221] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:22.774929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3a37e-cf40-4223-8cf8-fafd9985cb67 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:22.882672 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3a37e-cf40-4223-8cf8-fafd9985cb67 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:22.915730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0fb64-b279-4601-a039-33c37b57c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:22.916343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0fb64-b279-4601-a039-33c37b57c626 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:22.916614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0fb64-b279-4601-a039-33c37b57c626 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:22.916902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0fb64-b279-4601-a039-33c37b57c626 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:22.926641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0fb64-b279-4601-a039-33c37b57c626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8cb0fb64-b279-4601-a039-33c37b57c626 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:15:22.931815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:22.933009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:22.934106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 617/1222] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:15:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:22.938984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05159bec-8bc5-4c33-b9e0-8dceb82751b4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:23.042017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05159bec-8bc5-4c33-b9e0-8dceb82751b4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:23.065136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:23.075964 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e20956-e3a6-400d-847e-8ec01532d0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:23.076552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e20956-e3a6-400d-847e-8ec01532d0cb None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:23.076847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e20956-e3a6-400d-847e-8ec01532d0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:23.077118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e20956-e3a6-400d-847e-8ec01532d0cb None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:23.077757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e20956-e3a6-400d-847e-8ec01532d0cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:23.078050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e20956-e3a6-400d-847e-8ec01532d0cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:23.079393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e20956-e3a6-400d-847e-8ec01532d0cb None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:23.087793 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 606/1223] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:15:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:23.093625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0fb64-b279-4601-a039-33c37b57c626 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:23.137779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:23.191148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0fb64-b279-4601-a039-33c37b57c626 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:23.224035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742557dd-c9e7-44fb-a425-ac7ef42d8c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:23.224692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742557dd-c9e7-44fb-a425-ac7ef42d8c4f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:23.224982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742557dd-c9e7-44fb-a425-ac7ef42d8c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:23.225251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742557dd-c9e7-44fb-a425-ac7ef42d8c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:23.234762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742557dd-c9e7-44fb-a425-ac7ef42d8c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:23.236441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742557dd-c9e7-44fb-a425-ac7ef42d8c4f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-742557dd-c9e7-44fb-a425-ac7ef42d8c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:23.250765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:23.251883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:23.252841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:23.252841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:23.252841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:23.252841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 618/1224] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:23.258392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e20956-e3a6-400d-847e-8ec01532d0cb None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:23.358050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e20956-e3a6-400d-847e-8ec01532d0cb None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:23.391494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1629e-d262-4654-b22a-83d01e834dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:23.392082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1629e-d262-4654-b22a-83d01e834dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:23.392354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1629e-d262-4654-b22a-83d01e834dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:23.392623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1629e-d262-4654-b22a-83d01e834dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:23.393270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1629e-d262-4654-b22a-83d01e834dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:23.393570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1629e-d262-4654-b22a-83d01e834dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:23.394911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1629e-d262-4654-b22a-83d01e834dc1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:23.407345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 607/1225] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:15:23.413674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742557dd-c9e7-44fb-a425-ac7ef42d8c4f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:23.507371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:23.515720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742557dd-c9e7-44fb-a425-ac7ef42d8c4f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:23.553534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9dbce1-68c7-4eb1-8ec5-968b6c991dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:23.554150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9dbce1-68c7-4eb1-8ec5-968b6c991dac None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:23.554426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9dbce1-68c7-4eb1-8ec5-968b6c991dac None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:23.554695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9dbce1-68c7-4eb1-8ec5-968b6c991dac None admin] PATH_INFO: `/v3/users/placement` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:23.562114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9dbce1-68c7-4eb1-8ec5-968b6c991dac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:23.563658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9dbce1-68c7-4eb1-8ec5-968b6c991dac None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8c9dbce1-68c7-4eb1-8ec5-968b6c991dac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:15:23.575770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:23.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:23.577802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:23.579343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 619/1226] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 11:15:23.582780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1629e-d262-4654-b22a-83d01e834dc1 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:23.630386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:23.684315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1629e-d262-4654-b22a-83d01e834dc1 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:23.717502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f825d67-8d96-48bd-8616-679dc15fac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:23.718089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f825d67-8d96-48bd-8616-679dc15fac7e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:23.718359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f825d67-8d96-48bd-8616-679dc15fac7e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:23.718620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f825d67-8d96-48bd-8616-679dc15fac7e None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:23.719326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f825d67-8d96-48bd-8616-679dc15fac7e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:23.719623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f825d67-8d96-48bd-8616-679dc15fac7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:23.721298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f825d67-8d96-48bd-8616-679dc15fac7e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:23.739721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 608/1227] 10.222.0.84 () {56 vars in 1143 bytes} [Wed Aug 24 11:15:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:23.747328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9dbce1-68c7-4eb1-8ec5-968b6c991dac None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:23.862280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9dbce1-68c7-4eb1-8ec5-968b6c991dac None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:23.894875 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7960f2-037a-4c27-a685-f2ce538c6210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:23.895436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7960f2-037a-4c27-a685-f2ce538c6210 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:23.895709 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7960f2-037a-4c27-a685-f2ce538c6210 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:23.895974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7960f2-037a-4c27-a685-f2ce538c6210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:23.906857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7960f2-037a-4c27-a685-f2ce538c6210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:23.908283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7960f2-037a-4c27-a685-f2ce538c6210 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-eb7960f2-037a-4c27-a685-f2ce538c6210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:23.922420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:23.923551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:23.924443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:23.924443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:23.924443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:23.924443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 620/1228] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:23.929706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f825d67-8d96-48bd-8616-679dc15fac7e None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:24.029187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f825d67-8d96-48bd-8616-679dc15fac7e None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:24.031452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:24.061944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6b4ab-3327-4bff-b682-7eb4b7950a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:24.062529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6b4ab-3327-4bff-b682-7eb4b7950a0c None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:24.062801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6b4ab-3327-4bff-b682-7eb4b7950a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:24.063067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6b4ab-3327-4bff-b682-7eb4b7950a0c None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:24.063725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6b4ab-3327-4bff-b682-7eb4b7950a0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:24.064018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6b4ab-3327-4bff-b682-7eb4b7950a0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:24.065391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6b4ab-3327-4bff-b682-7eb4b7950a0c None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:24.076644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 609/1229] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:24.082360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7960f2-037a-4c27-a685-f2ce538c6210 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:24.092422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:24.182319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7960f2-037a-4c27-a685-f2ce538c6210 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:24.220434 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d8ad5-f938-4efc-a1c7-fdc1b1f8b5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:24.221082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d8ad5-f938-4efc-a1c7-fdc1b1f8b5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:24.221358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d8ad5-f938-4efc-a1c7-fdc1b1f8b5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:24.221625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d8ad5-f938-4efc-a1c7-fdc1b1f8b5f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:24.228189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d8ad5-f938-4efc-a1c7-fdc1b1f8b5f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:24.229650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d8ad5-f938-4efc-a1c7-fdc1b1f8b5f3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-c40d8ad5-f938-4efc-a1c7-fdc1b1f8b5f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:15:24.241022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:24.241966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:24.242971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:15:24.242971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:15:24.242971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:15:24.242971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:15:24.242971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:15:24.242971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:15:24.242971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:24.242971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:24.242971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 621/1230] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:24.250303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6b4ab-3327-4bff-b682-7eb4b7950a0c None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:24.349936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6b4ab-3327-4bff-b682-7eb4b7950a0c None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:24.383149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28bee4-01a6-41cc-9b6c-2f66e0b81d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:24.383750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28bee4-01a6-41cc-9b6c-2f66e0b81d52 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:24.384023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28bee4-01a6-41cc-9b6c-2f66e0b81d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:24.384293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28bee4-01a6-41cc-9b6c-2f66e0b81d52 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:24.384997 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28bee4-01a6-41cc-9b6c-2f66e0b81d52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:24.385302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28bee4-01a6-41cc-9b6c-2f66e0b81d52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:24.386724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28bee4-01a6-41cc-9b6c-2f66e0b81d52 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:24.399058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 610/1231] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:24.405474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d8ad5-f938-4efc-a1c7-fdc1b1f8b5f3 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:24.471006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:24.504999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d8ad5-f938-4efc-a1c7-fdc1b1f8b5f3 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:24.538289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6cbae-0fcf-4251-b459-052d507043c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:24.538939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6cbae-0fcf-4251-b459-052d507043c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:24.539216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6cbae-0fcf-4251-b459-052d507043c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:24.539485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6cbae-0fcf-4251-b459-052d507043c9 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/c7cc35f5ddf0446e90ded5c85f21f37d/roles/69cd1c433d70467a91f4a9f47382a412` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:24.555417 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6cbae-0fcf-4251-b459-052d507043c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=c7cc35f5ddf0446e90ded5c85f21f37d, role_id=69cd1c433d70467a91f4a9f47382a412)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:24.559724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6cbae-0fcf-4251-b459-052d507043c9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:24.682811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 622/1232] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:15:24 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/c7cc35f5ddf0446e90ded5c85f21f37d/roles/69cd1c433d70467a91f4a9f47382a412 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:26.572040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bab5f0-a1dc-46c4-b570-632f9394be22 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:26.572599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bab5f0-a1dc-46c4-b570-632f9394be22 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:26.572599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bab5f0-a1dc-46c4-b570-632f9394be22 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:26.573982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 611/1233] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:26.579467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0a3b3-f68d-4f6c-afb8-b00cb619351c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:26.579886 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0a3b3-f68d-4f6c-afb8-b00cb619351c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:26.579997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0a3b3-f68d-4f6c-afb8-b00cb619351c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:26.598959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9ca0a3b3-f68d-4f6c-afb8-b00cb619351c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:26.600181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9ca0a3b3-f68d-4f6c-afb8-b00cb619351c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:26.666683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 623/1234] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:26.939084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bab5f0-a1dc-46c4-b570-632f9394be22 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:26.964552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:27.039826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bab5f0-a1dc-46c4-b570-632f9394be22 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:27.074145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14e8b-d337-4d32-a742-92ced219a614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:27.074780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14e8b-d337-4d32-a742-92ced219a614 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:27.075039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14e8b-d337-4d32-a742-92ced219a614 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:27.075308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14e8b-d337-4d32-a742-92ced219a614 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:27.085482 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a14e8b-d337-4d32-a742-92ced219a614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-d1a14e8b-d337-4d32-a742-92ced219a614 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:15:27.090112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:27.091313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:27.092199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 612/1235] 10.222.0.84 () {56 vars in 1102 bytes} [Wed Aug 24 11:15:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:27.097233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0a3b3-f68d-4f6c-afb8-b00cb619351c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:27.143085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:27.195230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0a3b3-f68d-4f6c-afb8-b00cb619351c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:27.228990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38aeb29-44f7-4eae-8119-de3017dd6bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:27.229857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38aeb29-44f7-4eae-8119-de3017dd6bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:27.230148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38aeb29-44f7-4eae-8119-de3017dd6bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:27.230607 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38aeb29-44f7-4eae-8119-de3017dd6bd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:27.231303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38aeb29-44f7-4eae-8119-de3017dd6bd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:27.231952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38aeb29-44f7-4eae-8119-de3017dd6bd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:27.233395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38aeb29-44f7-4eae-8119-de3017dd6bd9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:27.242531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 624/1236] 10.222.0.84 () {56 vars in 1103 bytes} [Wed Aug 24 11:15:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:27.248298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14e8b-d337-4d32-a742-92ced219a614 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:27.339116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:27.357171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14e8b-d337-4d32-a742-92ced219a614 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:27.391327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3767391-e917-45af-be42-ad21264fb3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:27.391941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3767391-e917-45af-be42-ad21264fb3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:27.392212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3767391-e917-45af-be42-ad21264fb3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:27.392482 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3767391-e917-45af-be42-ad21264fb3a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:27.403264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3767391-e917-45af-be42-ad21264fb3a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:27.404691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3767391-e917-45af-be42-ad21264fb3a9 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-b3767391-e917-45af-be42-ad21264fb3a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:27.419413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:27.420519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:27.421626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:27.421626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:27.421626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:27.421626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 613/1237] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:27.426453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38aeb29-44f7-4eae-8119-de3017dd6bd9 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:27.543021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38aeb29-44f7-4eae-8119-de3017dd6bd9 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:27.552202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:27.578164 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca04ca-8ad7-4696-9430-e79e17c37a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:27.578770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca04ca-8ad7-4696-9430-e79e17c37a3c None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:27.579045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca04ca-8ad7-4696-9430-e79e17c37a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:27.579307 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca04ca-8ad7-4696-9430-e79e17c37a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:27.579934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca04ca-8ad7-4696-9430-e79e17c37a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:27.580484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca04ca-8ad7-4696-9430-e79e17c37a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:27.581610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca04ca-8ad7-4696-9430-e79e17c37a3c None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:27.593970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 625/1238] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:27.600112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3767391-e917-45af-be42-ad21264fb3a9 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:27.715828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3767391-e917-45af-be42-ad21264fb3a9 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:27.751843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc58fc6-e954-4b95-bfe9-8192a8e93718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:27.752465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc58fc6-e954-4b95-bfe9-8192a8e93718 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:27.752740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc58fc6-e954-4b95-bfe9-8192a8e93718 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:27.753041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc58fc6-e954-4b95-bfe9-8192a8e93718 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:27.756364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:27.762259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc58fc6-e954-4b95-bfe9-8192a8e93718 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:27.763700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc58fc6-e954-4b95-bfe9-8192a8e93718 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-5cc58fc6-e954-4b95-bfe9-8192a8e93718 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:15:27.775295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:27.779484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:27.781281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 614/1239] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:27.782874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca04ca-8ad7-4696-9430-e79e17c37a3c None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:27.895601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca04ca-8ad7-4696-9430-e79e17c37a3c None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:27.933222 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acd00b-a88c-45be-9013-058054469cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:27.934195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acd00b-a88c-45be-9013-058054469cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:27.934679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acd00b-a88c-45be-9013-058054469cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:27.935129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21acd00b-a88c-45be-9013-058054469cd3 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:27.936380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21acd00b-a88c-45be-9013-058054469cd3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:27.936931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21acd00b-a88c-45be-9013-058054469cd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:27.939539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21acd00b-a88c-45be-9013-058054469cd3 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:27.966470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 626/1240] 10.222.0.84 () {56 vars in 1143 bytes} [Wed Aug 24 11:15:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:15:27.972742 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc58fc6-e954-4b95-bfe9-8192a8e93718 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:28.089933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc58fc6-e954-4b95-bfe9-8192a8e93718 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:28.126741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c15e6a-adcd-4421-a01a-94b3cec97738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:28.127350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c15e6a-adcd-4421-a01a-94b3cec97738 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:28.127623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c15e6a-adcd-4421-a01a-94b3cec97738 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:28.127918 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c15e6a-adcd-4421-a01a-94b3cec97738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:28.201786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:28.209334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c15e6a-adcd-4421-a01a-94b3cec97738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:28.211106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c15e6a-adcd-4421-a01a-94b3cec97738 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:28.321999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 615/1241] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:15:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:15:28.328150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acd00b-a88c-45be-9013-058054469cd3 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:28.355888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:28.428692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21acd00b-a88c-45be-9013-058054469cd3 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:28.461729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c549ad5-5d28-47e7-87e2-346b944339ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:28.462276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c549ad5-5d28-47e7-87e2-346b944339ec None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:28.462549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c549ad5-5d28-47e7-87e2-346b944339ec None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:28.462812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c549ad5-5d28-47e7-87e2-346b944339ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:28.472873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c549ad5-5d28-47e7-87e2-346b944339ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:28.474361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c549ad5-5d28-47e7-87e2-346b944339ec None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-9c549ad5-5d28-47e7-87e2-346b944339ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:28.490490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:28.491568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:28.492542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:28.492542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:28.492542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:28.492542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 627/1242] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:28.497949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c15e6a-adcd-4421-a01a-94b3cec97738 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:28.591713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:28.598773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c15e6a-adcd-4421-a01a-94b3cec97738 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:28.635367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed84f85-42f9-4d3a-9eda-b78af24c5a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:28.636026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed84f85-42f9-4d3a-9eda-b78af24c5a29 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:28.636299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed84f85-42f9-4d3a-9eda-b78af24c5a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:28.636622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed84f85-42f9-4d3a-9eda-b78af24c5a29 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:28.637295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed84f85-42f9-4d3a-9eda-b78af24c5a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:28.637599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed84f85-42f9-4d3a-9eda-b78af24c5a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:28.638955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed84f85-42f9-4d3a-9eda-b78af24c5a29 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:28.652840 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 616/1243] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:28.658905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c549ad5-5d28-47e7-87e2-346b944339ec None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:28.744208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:28.764376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c549ad5-5d28-47e7-87e2-346b944339ec None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:28.804073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13da85b-c639-4c67-82c5-224d29766543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:28.804974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13da85b-c639-4c67-82c5-224d29766543 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:28.805223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13da85b-c639-4c67-82c5-224d29766543 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:28.805317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13da85b-c639-4c67-82c5-224d29766543 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:28.812453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13da85b-c639-4c67-82c5-224d29766543 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:28.813926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13da85b-c639-4c67-82c5-224d29766543 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-a13da85b-c639-4c67-82c5-224d29766543 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:15:28.826973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:28.829479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:28.830628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:15:28.830628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:15:28.830628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:15:28.830628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:15:28.830628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:15:28.830628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:15:28.830628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:28.830628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:28.830628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 628/1244] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:28.834529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed84f85-42f9-4d3a-9eda-b78af24c5a29 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:28.935028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed84f85-42f9-4d3a-9eda-b78af24c5a29 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:28.968281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6137d-2d0f-4415-823c-459671a650a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:28.968901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6137d-2d0f-4415-823c-459671a650a8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:28.969177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6137d-2d0f-4415-823c-459671a650a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:28.969445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6137d-2d0f-4415-823c-459671a650a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:28.970114 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6137d-2d0f-4415-823c-459671a650a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:28.970411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6137d-2d0f-4415-823c-459671a650a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:28.971811 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6137d-2d0f-4415-823c-459671a650a8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:28.983976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 617/1245] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:28.991542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13da85b-c639-4c67-82c5-224d29766543 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:29.091000 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13da85b-c639-4c67-82c5-224d29766543 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:29.124165 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd086e-5d55-4868-a81e-aa5235cfcc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:29.124755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd086e-5d55-4868-a81e-aa5235cfcc28 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:29.125061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd086e-5d55-4868-a81e-aa5235cfcc28 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:29.125328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd086e-5d55-4868-a81e-aa5235cfcc28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:29.126044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd086e-5d55-4868-a81e-aa5235cfcc28 None admin] RBAC: Adding query filter params (role.id=69cd1c433d70467a91f4a9f47382a412, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=c7cc35f5ddf0446e90ded5c85f21f37d) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:29.126345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd086e-5d55-4868-a81e-aa5235cfcc28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:29.127750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd086e-5d55-4868-a81e-aa5235cfcc28 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:29.133659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 629/1246] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:15:28 2022] GET /identity/v3/role_assignments?user.id=c7cc35f5ddf0446e90ded5c85f21f37d&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=69cd1c433d70467a91f4a9f47382a412 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:31.138820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286e3d9-a978-477c-8981-ef08e61ac129 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:31.139509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286e3d9-a978-477c-8981-ef08e61ac129 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:31.139509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286e3d9-a978-477c-8981-ef08e61ac129 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:31.140692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 618/1247] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:31.145831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4c5c6-9ee7-4d71-a1ce-294b9db090bf None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:31.146841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4c5c6-9ee7-4d71-a1ce-294b9db090bf None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:31.147029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4c5c6-9ee7-4d71-a1ce-294b9db090bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:31.153873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:31.165939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-93f4c5c6-9ee7-4d71-a1ce-294b9db090bf None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:31.167178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-93f4c5c6-9ee7-4d71-a1ce-294b9db090bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:31.232123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 630/1248] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:31.498481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286e3d9-a978-477c-8981-ef08e61ac129 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:31.516749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:31.598970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286e3d9-a978-477c-8981-ef08e61ac129 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:31.634053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe59dc-0fc0-4743-9f8a-1c79cf44fb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:31.634666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe59dc-0fc0-4743-9f8a-1c79cf44fb00 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:31.634939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe59dc-0fc0-4743-9f8a-1c79cf44fb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:31.635267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe59dc-0fc0-4743-9f8a-1c79cf44fb00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:31.645230 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe59dc-0fc0-4743-9f8a-1c79cf44fb00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-98fe59dc-0fc0-4743-9f8a-1c79cf44fb00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:15:31.650151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:31.653151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:31.654591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 619/1249] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:31.659702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4c5c6-9ee7-4d71-a1ce-294b9db090bf None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:31.756762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4c5c6-9ee7-4d71-a1ce-294b9db090bf None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:31.784755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:31.790221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db572593-3b1d-49a3-9bbb-10c39c6cd88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:31.790808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db572593-3b1d-49a3-9bbb-10c39c6cd88f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:31.791084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db572593-3b1d-49a3-9bbb-10c39c6cd88f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:31.791358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db572593-3b1d-49a3-9bbb-10c39c6cd88f None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:31.792001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db572593-3b1d-49a3-9bbb-10c39c6cd88f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:31.792295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db572593-3b1d-49a3-9bbb-10c39c6cd88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:31.793670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db572593-3b1d-49a3-9bbb-10c39c6cd88f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:31.801577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 631/1250] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:31.808460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe59dc-0fc0-4743-9f8a-1c79cf44fb00 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:31.884897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:31.909440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe59dc-0fc0-4743-9f8a-1c79cf44fb00 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:31.948090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d607fa-7848-4817-bc99-1d7ebbaec003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:31.948707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d607fa-7848-4817-bc99-1d7ebbaec003 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:31.949003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d607fa-7848-4817-bc99-1d7ebbaec003 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:31.949314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d607fa-7848-4817-bc99-1d7ebbaec003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:31.960785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d607fa-7848-4817-bc99-1d7ebbaec003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:31.962259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d607fa-7848-4817-bc99-1d7ebbaec003 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e9d607fa-7848-4817-bc99-1d7ebbaec003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:31.977585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:31.978779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:31.979860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:31.979860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:31.979860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:31.979860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 620/1251] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:31.984769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db572593-3b1d-49a3-9bbb-10c39c6cd88f None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:32.086224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db572593-3b1d-49a3-9bbb-10c39c6cd88f None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:32.120371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd76ca7-e64e-4b3e-a5ea-48fc5e286706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:32.120994 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd76ca7-e64e-4b3e-a5ea-48fc5e286706 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:32.121268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd76ca7-e64e-4b3e-a5ea-48fc5e286706 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:32.121532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd76ca7-e64e-4b3e-a5ea-48fc5e286706 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:32.122156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd76ca7-e64e-4b3e-a5ea-48fc5e286706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:32.122450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd76ca7-e64e-4b3e-a5ea-48fc5e286706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:32.123789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd76ca7-e64e-4b3e-a5ea-48fc5e286706 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:32.135908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 632/1252] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:32.141659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d607fa-7848-4817-bc99-1d7ebbaec003 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:32.241283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d607fa-7848-4817-bc99-1d7ebbaec003 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:32.266942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:32.276240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b26bb-a57c-4c89-8759-3709d8f36aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:32.276889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b26bb-a57c-4c89-8759-3709d8f36aec None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:32.277168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b26bb-a57c-4c89-8759-3709d8f36aec None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:32.277439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b26bb-a57c-4c89-8759-3709d8f36aec None admin] PATH_INFO: `/v3/users/placement` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:32.287379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b26bb-a57c-4c89-8759-3709d8f36aec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:32.288974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b26bb-a57c-4c89-8759-3709d8f36aec None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-030b26bb-a57c-4c89-8759-3709d8f36aec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:15:32.301142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:32.302157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:32.303340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 621/1253] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:32.308655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd76ca7-e64e-4b3e-a5ea-48fc5e286706 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:32.368754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:32.412151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd76ca7-e64e-4b3e-a5ea-48fc5e286706 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:32.447005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dad41-76c4-4031-bb7f-d7d1aef98839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:32.447603 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dad41-76c4-4031-bb7f-d7d1aef98839 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:32.447880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dad41-76c4-4031-bb7f-d7d1aef98839 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:32.448150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dad41-76c4-4031-bb7f-d7d1aef98839 None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:32.448897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dad41-76c4-4031-bb7f-d7d1aef98839 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:32.449203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dad41-76c4-4031-bb7f-d7d1aef98839 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:32.450628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6dad41-76c4-4031-bb7f-d7d1aef98839 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:32.468271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 633/1254] 10.222.0.84 () {56 vars in 1143 bytes} [Wed Aug 24 11:15:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:32.474051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b26bb-a57c-4c89-8759-3709d8f36aec None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:32.574393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b26bb-a57c-4c89-8759-3709d8f36aec None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:32.614166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d9fb-297d-43cb-b0b8-d530328d7f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:32.614780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d9fb-297d-43cb-b0b8-d530328d7f1d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:32.615552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d9fb-297d-43cb-b0b8-d530328d7f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:32.615832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d9fb-297d-43cb-b0b8-d530328d7f1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:32.674970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:32.687471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40d9fb-297d-43cb-b0b8-d530328d7f1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:32.688944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40d9fb-297d-43cb-b0b8-d530328d7f1d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:32.783290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 622/1255] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:15:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:15:32.789532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dad41-76c4-4031-bb7f-d7d1aef98839 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:32.887996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6dad41-76c4-4031-bb7f-d7d1aef98839 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:32.901484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:32.923321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a86f8-5b77-45a7-b8dc-1a9340cbc847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:32.923876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a86f8-5b77-45a7-b8dc-1a9340cbc847 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:32.924150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a86f8-5b77-45a7-b8dc-1a9340cbc847 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:32.924412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a86f8-5b77-45a7-b8dc-1a9340cbc847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:32.935277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a86f8-5b77-45a7-b8dc-1a9340cbc847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:32.936861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a86f8-5b77-45a7-b8dc-1a9340cbc847 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-e61a86f8-5b77-45a7-b8dc-1a9340cbc847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:32.951200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:32.956357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:32.957523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:32.957523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:32.957523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:32.957523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 634/1256] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:32.959591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d9fb-297d-43cb-b0b8-d530328d7f1d None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:33.045925 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:33.057733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d9fb-297d-43cb-b0b8-d530328d7f1d None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:33.091455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24433ec7-899b-4819-942e-11c1985e6fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:33.092109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24433ec7-899b-4819-942e-11c1985e6fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:33.092382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24433ec7-899b-4819-942e-11c1985e6fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:33.092650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24433ec7-899b-4819-942e-11c1985e6fe3 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:33.093312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24433ec7-899b-4819-942e-11c1985e6fe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:33.093610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24433ec7-899b-4819-942e-11c1985e6fe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:33.094960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24433ec7-899b-4819-942e-11c1985e6fe3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:33.106079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 623/1257] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:33.112049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a86f8-5b77-45a7-b8dc-1a9340cbc847 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:33.211979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a86f8-5b77-45a7-b8dc-1a9340cbc847 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:33.244684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8308276-7094-49f4-b19d-443a1505e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:33.245354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8308276-7094-49f4-b19d-443a1505e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:33.245628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8308276-7094-49f4-b19d-443a1505e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:33.245895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8308276-7094-49f4-b19d-443a1505e8d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:33.252379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8308276-7094-49f4-b19d-443a1505e8d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:33.253842 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8308276-7094-49f4-b19d-443a1505e8d9 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-c8308276-7094-49f4-b19d-443a1505e8d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:15:33.265194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:33.271783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:33.272866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:15:33.272866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:15:33.272866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:15:33.272866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:15:33.272866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:15:33.272866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:15:33.272866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:33.272866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:33.272866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 635/1258] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:33.273364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24433ec7-899b-4819-942e-11c1985e6fe3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:33.372216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24433ec7-899b-4819-942e-11c1985e6fe3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:33.406126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bcd38-3034-4f74-a655-6acd1cf82407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:33.406716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bcd38-3034-4f74-a655-6acd1cf82407 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:33.406989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bcd38-3034-4f74-a655-6acd1cf82407 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:33.407293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bcd38-3034-4f74-a655-6acd1cf82407 None admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:33.407967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bcd38-3034-4f74-a655-6acd1cf82407 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:33.408266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bcd38-3034-4f74-a655-6acd1cf82407 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:33.409848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bcd38-3034-4f74-a655-6acd1cf82407 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:33.421752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 624/1259] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:15:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:33.428028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8308276-7094-49f4-b19d-443a1505e8d9 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:33.455962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:33.529369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8308276-7094-49f4-b19d-443a1505e8d9 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:33.563237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f47f50-d3a4-4ed5-9e6c-79674ead0c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:33.563830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f47f50-d3a4-4ed5-9e6c-79674ead0c96 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:33.564104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f47f50-d3a4-4ed5-9e6c-79674ead0c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:33.564372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f47f50-d3a4-4ed5-9e6c-79674ead0c96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:33.565111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f47f50-d3a4-4ed5-9e6c-79674ead0c96 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=c7cc35f5ddf0446e90ded5c85f21f37d) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:33.565410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f47f50-d3a4-4ed5-9e6c-79674ead0c96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:33.566867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f47f50-d3a4-4ed5-9e6c-79674ead0c96 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:33.572204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 636/1260] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:15:33 2022] GET /identity/v3/role_assignments?user.id=c7cc35f5ddf0446e90ded5c85f21f37d&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:35.463680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6377a-551c-4881-a9fb-cd5cd20ba206 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:35.464211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6377a-551c-4881-a9fb-cd5cd20ba206 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:35.464211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6377a-551c-4881-a9fb-cd5cd20ba206 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:35.465571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 625/1261] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:35.470538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daff215-6e54-47e1-b58c-b61638b58852 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:35.470946 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daff215-6e54-47e1-b58c-b61638b58852 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:35.471088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daff215-6e54-47e1-b58c-b61638b58852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:35.491466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9daff215-6e54-47e1-b58c-b61638b58852 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:35.492687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9daff215-6e54-47e1-b58c-b61638b58852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:35.559079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 637/1262] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:35.831771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6377a-551c-4881-a9fb-cd5cd20ba206 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:35.843591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:35.934296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6377a-551c-4881-a9fb-cd5cd20ba206 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:35.969927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd9b42-4e79-47cb-816d-e63d7c10b41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:35.970547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd9b42-4e79-47cb-816d-e63d7c10b41b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:35.970823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd9b42-4e79-47cb-816d-e63d7c10b41b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:35.971091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd9b42-4e79-47cb-816d-e63d7c10b41b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:35.981275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd9b42-4e79-47cb-816d-e63d7c10b41b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-bebd9b42-4e79-47cb-816d-e63d7c10b41b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:15:35.985987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:35.987542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:35.988406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 626/1263] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:15:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:35.997377 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daff215-6e54-47e1-b58c-b61638b58852 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:36.016394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:36.095310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daff215-6e54-47e1-b58c-b61638b58852 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:36.129468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379562e-922e-4e46-9a22-318c3ee27a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:36.130059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379562e-922e-4e46-9a22-318c3ee27a30 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:36.130416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379562e-922e-4e46-9a22-318c3ee27a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:36.130694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379562e-922e-4e46-9a22-318c3ee27a30 None admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:36.131340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379562e-922e-4e46-9a22-318c3ee27a30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:36.131677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379562e-922e-4e46-9a22-318c3ee27a30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:36.133048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379562e-922e-4e46-9a22-318c3ee27a30 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:36.140860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 638/1264] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:15:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:36.146920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd9b42-4e79-47cb-816d-e63d7c10b41b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:36.214022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:36.247166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd9b42-4e79-47cb-816d-e63d7c10b41b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:36.280050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022edffd-725c-4ce1-9939-196da8367f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:36.280669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022edffd-725c-4ce1-9939-196da8367f0b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:36.280976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022edffd-725c-4ce1-9939-196da8367f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:36.281303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022edffd-725c-4ce1-9939-196da8367f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:36.292539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022edffd-725c-4ce1-9939-196da8367f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:36.294076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022edffd-725c-4ce1-9939-196da8367f0b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-022edffd-725c-4ce1-9939-196da8367f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:36.309069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:36.310096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:36.311006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:36.311006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:36.311006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:36.311006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 627/1265] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:36.316093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379562e-922e-4e46-9a22-318c3ee27a30 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:36.400113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:36.418819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379562e-922e-4e46-9a22-318c3ee27a30 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:36.451104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b724740-b814-43f1-b129-54fb96315618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:36.451692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b724740-b814-43f1-b129-54fb96315618 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:36.451966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b724740-b814-43f1-b129-54fb96315618 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:36.452230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b724740-b814-43f1-b129-54fb96315618 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:36.452880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b724740-b814-43f1-b129-54fb96315618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:36.453177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b724740-b814-43f1-b129-54fb96315618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:36.454515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b724740-b814-43f1-b129-54fb96315618 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:36.466253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 639/1266] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:36.472234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022edffd-725c-4ce1-9939-196da8367f0b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:36.573310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022edffd-725c-4ce1-9939-196da8367f0b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:36.592920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:36.607188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63438c15-0f3e-416e-949b-e2585c87625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:36.607814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438c15-0f3e-416e-949b-e2585c87625b None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:36.608084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438c15-0f3e-416e-949b-e2585c87625b None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:36.608353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438c15-0f3e-416e-949b-e2585c87625b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:36.616170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63438c15-0f3e-416e-949b-e2585c87625b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:36.617617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63438c15-0f3e-416e-949b-e2585c87625b None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-63438c15-0f3e-416e-949b-e2585c87625b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:15:36.628450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:36.634417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:36.635555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 628/1267] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:36.636270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b724740-b814-43f1-b129-54fb96315618 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:36.733751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b724740-b814-43f1-b129-54fb96315618 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:36.767065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b8856-8970-42de-a465-abb8cbcc61ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:36.767660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b8856-8970-42de-a465-abb8cbcc61ff None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:36.767927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b8856-8970-42de-a465-abb8cbcc61ff None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:36.768190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b8856-8970-42de-a465-abb8cbcc61ff None admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:36.768930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b8856-8970-42de-a465-abb8cbcc61ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:36.771580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b8856-8970-42de-a465-abb8cbcc61ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:36.771580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b8856-8970-42de-a465-abb8cbcc61ff None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:36.779657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:36.787878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 640/1268] 10.222.0.84 () {56 vars in 1143 bytes} [Wed Aug 24 11:15:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:36.793855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63438c15-0f3e-416e-949b-e2585c87625b None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:36.894374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63438c15-0f3e-416e-949b-e2585c87625b None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:36.926934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66da8e-fc41-44cc-8757-87146555284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:36.928032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66da8e-fc41-44cc-8757-87146555284a None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:36.928307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66da8e-fc41-44cc-8757-87146555284a None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:36.928582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66da8e-fc41-44cc-8757-87146555284a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:36.938842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66da8e-fc41-44cc-8757-87146555284a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:36.940271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66da8e-fc41-44cc-8757-87146555284a None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-dd66da8e-fc41-44cc-8757-87146555284a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:36.954755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:36.955935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:36.957132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:36.957132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:36.957132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:36.957132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 629/1269] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:36.962764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b8856-8970-42de-a465-abb8cbcc61ff None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:37.062921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b8856-8970-42de-a465-abb8cbcc61ff None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:37.097123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780a90d-0415-4519-b01d-cf496d55d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:37.097719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780a90d-0415-4519-b01d-cf496d55d523 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:37.097999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780a90d-0415-4519-b01d-cf496d55d523 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:37.098270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780a90d-0415-4519-b01d-cf496d55d523 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:37.098900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780a90d-0415-4519-b01d-cf496d55d523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:37.099252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780a90d-0415-4519-b01d-cf496d55d523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:37.100788 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780a90d-0415-4519-b01d-cf496d55d523 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:37.113184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 641/1270] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:37.119120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66da8e-fc41-44cc-8757-87146555284a None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:37.151588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:37.224776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66da8e-fc41-44cc-8757-87146555284a None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:37.259522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc848a9a-342f-4376-96fa-0ee8b77f9329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:37.274218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc848a9a-342f-4376-96fa-0ee8b77f9329 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:37.274218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc848a9a-342f-4376-96fa-0ee8b77f9329 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:37.274218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc848a9a-342f-4376-96fa-0ee8b77f9329 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:37.274218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc848a9a-342f-4376-96fa-0ee8b77f9329 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:37.274218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc848a9a-342f-4376-96fa-0ee8b77f9329 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-fc848a9a-342f-4376-96fa-0ee8b77f9329 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:15:37.280083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:37.281182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:37.282382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:15:37.282382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:15:37.282382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:15:37.282382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:15:37.282382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:15:37.282382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:15:37.282382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:37.282382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:37.282382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 630/1271] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:37.287148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780a90d-0415-4519-b01d-cf496d55d523 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:37.345836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:37.387637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780a90d-0415-4519-b01d-cf496d55d523 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:37.420048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f4d6b-af13-4590-bc28-cc52e3db8602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:37.422343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f4d6b-af13-4590-bc28-cc52e3db8602 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:37.422343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f4d6b-af13-4590-bc28-cc52e3db8602 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:37.422343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f4d6b-af13-4590-bc28-cc52e3db8602 None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:37.422343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f4d6b-af13-4590-bc28-cc52e3db8602 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:37.422343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f4d6b-af13-4590-bc28-cc52e3db8602 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:37.423619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f4d6b-af13-4590-bc28-cc52e3db8602 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:37.436599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 642/1272] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:37.442707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc848a9a-342f-4376-96fa-0ee8b77f9329 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:37.534503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:37.543702 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc848a9a-342f-4376-96fa-0ee8b77f9329 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:37.576707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b48dea-aa08-4d9e-bcd0-8c64e6eb2e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:37.577371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48dea-aa08-4d9e-bcd0-8c64e6eb2e47 None admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:37.577640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48dea-aa08-4d9e-bcd0-8c64e6eb2e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:37.577903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b48dea-aa08-4d9e-bcd0-8c64e6eb2e47 None admin] PATH_INFO: `/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/c7cc35f5ddf0446e90ded5c85f21f37d/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:37.579724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b48dea-aa08-4d9e-bcd0-8c64e6eb2e47 None admin] RBAC: Authorizing `identity:create_grant(project_id=93f8726a7b3b483aa8f337bd3466d23f, user_id=c7cc35f5ddf0446e90ded5c85f21f37d, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:37.581166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b48dea-aa08-4d9e-bcd0-8c64e6eb2e47 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:37.733646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 631/1273] 10.222.0.84 () {58 vars in 1439 bytes} [Wed Aug 24 11:15:37 2022] PUT /identity/v3/projects/93f8726a7b3b483aa8f337bd3466d23f/users/c7cc35f5ddf0446e90ded5c85f21f37d/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:15:39.635468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a95e6-d5ad-4e5a-90e1-06eeb785e1a8 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:39.636022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a95e6-d5ad-4e5a-90e1-06eeb785e1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:39.636132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a95e6-d5ad-4e5a-90e1-06eeb785e1a8 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:39.637579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 643/1274] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:39.643057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fc385-5b64-47c7-a7f7-9eda06bb4ec5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:39.643452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fc385-5b64-47c7-a7f7-9eda06bb4ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:39.643604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fc385-5b64-47c7-a7f7-9eda06bb4ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:39.661912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-845fc385-5b64-47c7-a7f7-9eda06bb4ec5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:39.663074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-845fc385-5b64-47c7-a7f7-9eda06bb4ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:39.735720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 632/1275] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:40.002814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a95e6-d5ad-4e5a-90e1-06eeb785e1a8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:40.101036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a95e6-d5ad-4e5a-90e1-06eeb785e1a8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:40.118554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:40.134600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badcb688-0007-430a-afa0-9953a9b579c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:40.135211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcb688-0007-430a-afa0-9953a9b579c8 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.135485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcb688-0007-430a-afa0-9953a9b579c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.135755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcb688-0007-430a-afa0-9953a9b579c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.145490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badcb688-0007-430a-afa0-9953a9b579c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-badcb688-0007-430a-afa0-9953a9b579c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 11:15:40.150318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:40.151426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:40.152442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 644/1276] 10.222.0.84 () {56 vars in 1096 bytes} [Wed Aug 24 11:15:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:40.157442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fc385-5b64-47c7-a7f7-9eda06bb4ec5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:40.236967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:40.255197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845fc385-5b64-47c7-a7f7-9eda06bb4ec5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:40.290677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c755a-50d0-4abe-8fc6-d9aec1e8fbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:40.291278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c755a-50d0-4abe-8fc6-d9aec1e8fbab None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.291548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c755a-50d0-4abe-8fc6-d9aec1e8fbab None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.291853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c755a-50d0-4abe-8fc6-d9aec1e8fbab None admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.292498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c755a-50d0-4abe-8fc6-d9aec1e8fbab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:40.292810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c755a-50d0-4abe-8fc6-d9aec1e8fbab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:40.294163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c755a-50d0-4abe-8fc6-d9aec1e8fbab None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:40.302594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 633/1277] 10.222.0.84 () {56 vars in 1099 bytes} [Wed Aug 24 11:15:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:40.308816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badcb688-0007-430a-afa0-9953a9b579c8 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:40.420140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badcb688-0007-430a-afa0-9953a9b579c8 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:40.454492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36183d0-7429-49fb-b050-5cf623b97bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:40.455110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36183d0-7429-49fb-b050-5cf623b97bca None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.455381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36183d0-7429-49fb-b050-5cf623b97bca None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.455643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36183d0-7429-49fb-b050-5cf623b97bca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.465610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36183d0-7429-49fb-b050-5cf623b97bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:40.467029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36183d0-7429-49fb-b050-5cf623b97bca None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-c36183d0-7429-49fb-b050-5cf623b97bca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:40.481247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:40.482378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:40.483286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:40.483286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:40.483286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:40.483286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 645/1278] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:40.488572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c755a-50d0-4abe-8fc6-d9aec1e8fbab None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:40.589027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c755a-50d0-4abe-8fc6-d9aec1e8fbab None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:40.623002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ceb9b-07ef-4c3a-b1de-4daf4527c3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:40.623598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ceb9b-07ef-4c3a-b1de-4daf4527c3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.623881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ceb9b-07ef-4c3a-b1de-4daf4527c3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.624158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ceb9b-07ef-4c3a-b1de-4daf4527c3e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.624817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ceb9b-07ef-4c3a-b1de-4daf4527c3e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:40.632726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ceb9b-07ef-4c3a-b1de-4daf4527c3e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:40.632726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ceb9b-07ef-4c3a-b1de-4daf4527c3e3 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:40.638053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 634/1279] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:40.643668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36183d0-7429-49fb-b050-5cf623b97bca None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:40.675161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:40.743320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36183d0-7429-49fb-b050-5cf623b97bca None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:40.776588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46270874-3120-4d93-a170-0fb701feab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:40.777262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46270874-3120-4d93-a170-0fb701feab01 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.777542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46270874-3120-4d93-a170-0fb701feab01 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.777811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46270874-3120-4d93-a170-0fb701feab01 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.785157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46270874-3120-4d93-a170-0fb701feab01 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:40.787333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46270874-3120-4d93-a170-0fb701feab01 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-46270874-3120-4d93-a170-0fb701feab01 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 11:15:40.798551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:40.799677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:40.800618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 646/1280] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:40.806164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ceb9b-07ef-4c3a-b1de-4daf4527c3e3 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:40.810012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:40.903671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ceb9b-07ef-4c3a-b1de-4daf4527c3e3 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:40.937645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35196b-7d81-4b4c-944a-6275418faa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:40.938239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35196b-7d81-4b4c-944a-6275418faa75 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:40.938514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35196b-7d81-4b4c-944a-6275418faa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:40.938842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35196b-7d81-4b4c-944a-6275418faa75 None admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:40.939549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35196b-7d81-4b4c-944a-6275418faa75 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:40.939852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35196b-7d81-4b4c-944a-6275418faa75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:40.941337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35196b-7d81-4b4c-944a-6275418faa75 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:40.959238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 635/1281] 10.222.0.84 () {56 vars in 1143 bytes} [Wed Aug 24 11:15:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:40.965079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46270874-3120-4d93-a170-0fb701feab01 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.056315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:41.065477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46270874-3120-4d93-a170-0fb701feab01 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.097609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f23698-6492-44b2-ab7d-c21972b389cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.098202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f23698-6492-44b2-ab7d-c21972b389cb None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.098475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f23698-6492-44b2-ab7d-c21972b389cb None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.098740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f23698-6492-44b2-ab7d-c21972b389cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.164413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f23698-6492-44b2-ab7d-c21972b389cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.165805 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f23698-6492-44b2-ab7d-c21972b389cb None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.254183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:41.264137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 647/1282] 10.222.0.84 () {58 vars in 1282 bytes} [Wed Aug 24 11:15:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 11:15:41.269979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35196b-7d81-4b4c-944a-6275418faa75 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.332607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:41.370046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35196b-7d81-4b4c-944a-6275418faa75 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.405335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e2691-fac1-4ca7-87cd-ff550e12c15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.405960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e2691-fac1-4ca7-87cd-ff550e12c15f None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.406234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e2691-fac1-4ca7-87cd-ff550e12c15f None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.406530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e2691-fac1-4ca7-87cd-ff550e12c15f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.416213 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e2691-fac1-4ca7-87cd-ff550e12c15f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.417627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e2691-fac1-4ca7-87cd-ff550e12c15f None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-e29e2691-fac1-4ca7-87cd-ff550e12c15f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 11:15:41.431594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:41.432767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 11:15:41.433699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 11:15:41.433699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 11:15:41.433699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:41.433699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 636/1283] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:41.438395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f23698-6492-44b2-ab7d-c21972b389cb None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.541097 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f23698-6492-44b2-ab7d-c21972b389cb None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.575115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88d68e-4a84-4973-91b2-96c9b3df3ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.575703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88d68e-4a84-4973-91b2-96c9b3df3ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.575980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88d68e-4a84-4973-91b2-96c9b3df3ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.576249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88d68e-4a84-4973-91b2-96c9b3df3ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.576908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88d68e-4a84-4973-91b2-96c9b3df3ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:41.577210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88d68e-4a84-4973-91b2-96c9b3df3ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.578577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88d68e-4a84-4973-91b2-96c9b3df3ea4 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.590260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 648/1284] 10.222.0.84 () {56 vars in 1109 bytes} [Wed Aug 24 11:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:41.596011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e2691-fac1-4ca7-87cd-ff550e12c15f None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.701200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e2691-fac1-4ca7-87cd-ff550e12c15f None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.715399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:41.734970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5e18b-2e0e-4e2f-a646-68bd714997e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.735586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5e18b-2e0e-4e2f-a646-68bd714997e8 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.735858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5e18b-2e0e-4e2f-a646-68bd714997e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.736124 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5e18b-2e0e-4e2f-a646-68bd714997e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.743776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5e18b-2e0e-4e2f-a646-68bd714997e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.745314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5e18b-2e0e-4e2f-a646-68bd714997e8 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application [None req-6bb5e18b-2e0e-4e2f-a646-68bd714997e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 11:15:41.758561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:41.759586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:41.760566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 11:15:41.760566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 11:15:41.760566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 11:15:41.760566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 11:15:41.760566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 11:15:41.760566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 11:15:41.760566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 11:15:41.760566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: ERROR keystone.server.flask.application Aug 24 11:15:41.760566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 637/1285] 10.222.0.84 () {56 vars in 1111 bytes} [Wed Aug 24 11:15:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:41.766200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88d68e-4a84-4973-91b2-96c9b3df3ea4 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:41.821291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:41.875039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88d68e-4a84-4973-91b2-96c9b3df3ea4 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:41.911452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561349f-0425-4387-b399-22690f11703f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:41.912306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561349f-0425-4387-b399-22690f11703f None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:41.912740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561349f-0425-4387-b399-22690f11703f None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:41.913045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561349f-0425-4387-b399-22690f11703f None admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:41.913742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561349f-0425-4387-b399-22690f11703f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:41.914047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561349f-0425-4387-b399-22690f11703f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:41.915493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561349f-0425-4387-b399-22690f11703f None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:41.929037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 649/1286] 10.222.0.84 () {56 vars in 1148 bytes} [Wed Aug 24 11:15:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:41.935580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5e18b-2e0e-4e2f-a646-68bd714997e8 None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:42.045892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5e18b-2e0e-4e2f-a646-68bd714997e8 None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:42.087758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c80f21-6ec8-4e82-b8e7-b259aba1a9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:42.088365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c80f21-6ec8-4e82-b8e7-b259aba1a9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:42.088637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c80f21-6ec8-4e82-b8e7-b259aba1a9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:42.088999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c80f21-6ec8-4e82-b8e7-b259aba1a9a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:42.089709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c80f21-6ec8-4e82-b8e7-b259aba1a9a1 None admin] RBAC: Adding query filter params (role.id=e9c4afeedebc4a3db8d3713e9671a05c, scope.project.id=93f8726a7b3b483aa8f337bd3466d23f, user.id=c7cc35f5ddf0446e90ded5c85f21f37d) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:42.090013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c80f21-6ec8-4e82-b8e7-b259aba1a9a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:42.091423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c80f21-6ec8-4e82-b8e7-b259aba1a9a1 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:42.097164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 638/1287] 10.222.0.84 () {56 vars in 1374 bytes} [Wed Aug 24 11:15:41 2022] GET /identity/v3/role_assignments?user.id=c7cc35f5ddf0446e90ded5c85f21f37d&scope.project.id=93f8726a7b3b483aa8f337bd3466d23f&role.id=e9c4afeedebc4a3db8d3713e9671a05c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:44.049435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102c923-6a10-4ad9-801d-da58c710a62b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:44.049959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102c923-6a10-4ad9-801d-da58c710a62b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:44.049959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102c923-6a10-4ad9-801d-da58c710a62b None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:44.051288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 650/1288] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:44.056931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a4000-a6e2-4f16-9589-b8d4e4a83430 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:44.060985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a4000-a6e2-4f16-9589-b8d4e4a83430 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:44.060985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a4000-a6e2-4f16-9589-b8d4e4a83430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:44.076003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-765a4000-a6e2-4f16-9589-b8d4e4a83430 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:44.077221 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-765a4000-a6e2-4f16-9589-b8d4e4a83430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:44.106730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:44.144077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 639/1289] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:44.420134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102c923-6a10-4ad9-801d-da58c710a62b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:44.519497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102c923-6a10-4ad9-801d-da58c710a62b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:44.530009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:44.553982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f107afc-020f-46a0-8443-b333a31eb8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:44.554601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f107afc-020f-46a0-8443-b333a31eb8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:44.554879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f107afc-020f-46a0-8443-b333a31eb8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:44.555146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f107afc-020f-46a0-8443-b333a31eb8b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:44.555717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f107afc-020f-46a0-8443-b333a31eb8b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:44.557122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f107afc-020f-46a0-8443-b333a31eb8b0 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-6f107afc-020f-46a0-8443-b333a31eb8b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:44.565549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:44.566828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:44.567816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 651/1290] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:15:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:44.572917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a4000-a6e2-4f16-9589-b8d4e4a83430 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:44.669490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a4000-a6e2-4f16-9589-b8d4e4a83430 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:44.703751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeda6c-8297-4a07-a035-d1a44c13318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:44.704268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeda6c-8297-4a07-a035-d1a44c13318d None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:44.704538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeda6c-8297-4a07-a035-d1a44c13318d None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:44.704833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeda6c-8297-4a07-a035-d1a44c13318d None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:44.705438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeda6c-8297-4a07-a035-d1a44c13318d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:44.705736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeda6c-8297-4a07-a035-d1a44c13318d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:44.707101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeda6c-8297-4a07-a035-d1a44c13318d None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:44.712469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 640/1291] 10.222.0.84 () {56 vars in 1116 bytes} [Wed Aug 24 11:15:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:44.718654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f107afc-020f-46a0-8443-b333a31eb8b0 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:44.817809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f107afc-020f-46a0-8443-b333a31eb8b0 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:44.852549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e5f86-4d62-4aab-a111-cf8d5361b0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:44.853183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5f86-4d62-4aab-a111-cf8d5361b0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:44.853454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5f86-4d62-4aab-a111-cf8d5361b0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:44.853714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5f86-4d62-4aab-a111-cf8d5361b0a5 None admin] PATH_INFO: `/v3/services` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:44.854321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e5f86-4d62-4aab-a111-cf8d5361b0a5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:44.854612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e5f86-4d62-4aab-a111-cf8d5361b0a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:44.855946 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e5f86-4d62-4aab-a111-cf8d5361b0a5 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:44.860243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 652/1292] 10.222.0.84 () {56 vars in 1116 bytes} [Wed Aug 24 11:15:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:46.761307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614abed1-58f0-48ca-b014-92f3b1c7184b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:46.761832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614abed1-58f0-48ca-b014-92f3b1c7184b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:46.761832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614abed1-58f0-48ca-b014-92f3b1c7184b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:46.763182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 641/1293] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:46.768235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ee0a4-53dd-4a56-b3b2-c5548a5853c8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:46.768609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ee0a4-53dd-4a56-b3b2-c5548a5853c8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:46.768777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ee0a4-53dd-4a56-b3b2-c5548a5853c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:46.788070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c93ee0a4-53dd-4a56-b3b2-c5548a5853c8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:46.789287 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c93ee0a4-53dd-4a56-b3b2-c5548a5853c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:46.819100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:46.855375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 653/1294] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:47.128778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614abed1-58f0-48ca-b014-92f3b1c7184b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:47.160246 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:47.230672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614abed1-58f0-48ca-b014-92f3b1c7184b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:47.263704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f405587-f954-4e60-b225-098dc427b000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:47.264291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f405587-f954-4e60-b225-098dc427b000 None admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:47.264562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f405587-f954-4e60-b225-098dc427b000 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:47.264850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f405587-f954-4e60-b225-098dc427b000 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:47.265478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f405587-f954-4e60-b225-098dc427b000 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:47.266812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f405587-f954-4e60-b225-098dc427b000 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:47.322486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 642/1295] 10.222.0.84 () {60 vars in 1141 bytes} [Wed Aug 24 11:15:47 2022] POST /identity/v3/services => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:15:49.265643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57065c58-4dc6-4fc5-a82b-c04b03ecbd8c None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:49.266169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57065c58-4dc6-4fc5-a82b-c04b03ecbd8c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:49.266169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57065c58-4dc6-4fc5-a82b-c04b03ecbd8c None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:49.267517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 654/1296] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:49.273219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f5355-23dc-4eb1-8a46-73898a88482f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:49.273618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f5355-23dc-4eb1-8a46-73898a88482f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:49.273743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f5355-23dc-4eb1-8a46-73898a88482f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:49.291419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7b6f5355-23dc-4eb1-8a46-73898a88482f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:49.292626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7b6f5355-23dc-4eb1-8a46-73898a88482f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:49.363509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 643/1297] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:49.631065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57065c58-4dc6-4fc5-a82b-c04b03ecbd8c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:49.729658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57065c58-4dc6-4fc5-a82b-c04b03ecbd8c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:49.762746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217094c2-5449-4eba-8174-cf5de9fe49d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:49.763366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217094c2-5449-4eba-8174-cf5de9fe49d0 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:49.763643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217094c2-5449-4eba-8174-cf5de9fe49d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:49.763910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217094c2-5449-4eba-8174-cf5de9fe49d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:49.764478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217094c2-5449-4eba-8174-cf5de9fe49d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:49.765952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217094c2-5449-4eba-8174-cf5de9fe49d0 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-217094c2-5449-4eba-8174-cf5de9fe49d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:49.775095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:49.776253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:49.777263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 655/1298] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:15:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:49.782345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f5355-23dc-4eb1-8a46-73898a88482f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:49.800520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:49.878771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f5355-23dc-4eb1-8a46-73898a88482f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:49.913215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d67f50-5633-407c-9b5b-7c63001fd0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:49.913749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d67f50-5633-407c-9b5b-7c63001fd0df None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:49.914024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d67f50-5633-407c-9b5b-7c63001fd0df None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:49.914291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d67f50-5633-407c-9b5b-7c63001fd0df None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:49.914933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d67f50-5633-407c-9b5b-7c63001fd0df None admin] RBAC: Adding query filter params (name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:49.915232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d67f50-5633-407c-9b5b-7c63001fd0df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:49.916581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d67f50-5633-407c-9b5b-7c63001fd0df None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:49.921551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 644/1299] 10.222.0.84 () {56 vars in 1116 bytes} [Wed Aug 24 11:15:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:49.927870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217094c2-5449-4eba-8174-cf5de9fe49d0 None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:49.962352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:50.029726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217094c2-5449-4eba-8174-cf5de9fe49d0 None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.062376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489402ca-9af2-49ce-973b-a79523fa5149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.062995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489402ca-9af2-49ce-973b-a79523fa5149 None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.063276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489402ca-9af2-49ce-973b-a79523fa5149 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.063542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489402ca-9af2-49ce-973b-a79523fa5149 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.064225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489402ca-9af2-49ce-973b-a79523fa5149 None admin] RBAC: Adding query filter params (interface=public, service_id=fea1bb264c2241da94fe20a604695fe3, region_id=RegionOne) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:50.064527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489402ca-9af2-49ce-973b-a79523fa5149 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.065922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489402ca-9af2-49ce-973b-a79523fa5149 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.071401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 656/1300] 10.222.0.84 () {56 vars in 1251 bytes} [Wed Aug 24 11:15:49 2022] GET /identity/v3/endpoints?service_id=fea1bb264c2241da94fe20a604695fe3&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:50.077244 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d67f50-5633-407c-9b5b-7c63001fd0df None admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:50.157208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:50.175201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d67f50-5633-407c-9b5b-7c63001fd0df None admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:50.209721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc69ee-6032-4d0a-b7f1-1852bb95a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:50.210315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc69ee-6032-4d0a-b7f1-1852bb95a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:50.210588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc69ee-6032-4d0a-b7f1-1852bb95a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:50.210855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc69ee-6032-4d0a-b7f1-1852bb95a9a0 None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:50.211447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc69ee-6032-4d0a-b7f1-1852bb95a9a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:50.212862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc69ee-6032-4d0a-b7f1-1852bb95a9a0 None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:50.219520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 645/1301] 10.222.0.84 () {56 vars in 1087 bytes} [Wed Aug 24 11:15:50 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:15:52.156314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e8832-ad5a-4da1-a97b-ce52836d8cb7 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:52.156878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e8832-ad5a-4da1-a97b-ce52836d8cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:52.156878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e8832-ad5a-4da1-a97b-ce52836d8cb7 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:52.158200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 657/1302] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:15:52.163909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589d77c-6814-4026-83c8-4c8ead40fb42 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:52.164313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589d77c-6814-4026-83c8-4c8ead40fb42 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:52.164456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589d77c-6814-4026-83c8-4c8ead40fb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:52.182887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c589d77c-6814-4026-83c8-4c8ead40fb42 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:15:52.184045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c589d77c-6814-4026-83c8-4c8ead40fb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:15:52.251055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 646/1303] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:15:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 11:15:52.524849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e8832-ad5a-4da1-a97b-ce52836d8cb7 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:52.620316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:52.627018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e8832-ad5a-4da1-a97b-ce52836d8cb7 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:52.660702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90b402-c06c-47a0-987f-dbf468118f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:52.662973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90b402-c06c-47a0-987f-dbf468118f9e None admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:52.662973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90b402-c06c-47a0-987f-dbf468118f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:52.662973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90b402-c06c-47a0-987f-dbf468118f9e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:52.662973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90b402-c06c-47a0-987f-dbf468118f9e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:52.664118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90b402-c06c-47a0-987f-dbf468118f9e None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application [None req-8f90b402-c06c-47a0-987f-dbf468118f9e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 11:15:52.672675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 11:15:52.673668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application with Lock( Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self._enter() Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application return self.creator() Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application created_value = creator( Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: ERROR keystone.server.flask.application Aug 24 11:15:52.674497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 658/1304] 10.222.0.84 () {56 vars in 1117 bytes} [Wed Aug 24 11:15:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 11:15:52.679901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589d77c-6814-4026-83c8-4c8ead40fb42 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:52.756348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:15:52.779857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589d77c-6814-4026-83c8-4c8ead40fb42 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:52.813462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce917583-6e2f-4878-808d-f6515694328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:52.814057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce917583-6e2f-4878-808d-f6515694328e None admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:52.814334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce917583-6e2f-4878-808d-f6515694328e None admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:52.814601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce917583-6e2f-4878-808d-f6515694328e None admin] PATH_INFO: `/v3/services` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:52.815216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce917583-6e2f-4878-808d-f6515694328e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:15:52.815513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce917583-6e2f-4878-808d-f6515694328e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:52.816893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce917583-6e2f-4878-808d-f6515694328e None admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:52.822387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 647/1305] 10.222.0.84 () {56 vars in 1116 bytes} [Wed Aug 24 11:15:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:15:52.828358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90b402-c06c-47a0-987f-dbf468118f9e None admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:15:52.928432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90b402-c06c-47a0-987f-dbf468118f9e None admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:15:52.962175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d0019-5e2b-4701-ba3f-06c7c315c994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:15:52.962735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d0019-5e2b-4701-ba3f-06c7c315c994 None admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:15:52.963009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d0019-5e2b-4701-ba3f-06c7c315c994 None admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:15:52.963275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d0019-5e2b-4701-ba3f-06c7c315c994 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:15:52.963914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d0019-5e2b-4701-ba3f-06c7c315c994 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:15:52.965306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d0019-5e2b-4701-ba3f-06c7c315c994 None admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:15:53.047987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 659/1306] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:15:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:16:02.836506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=30 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:16:03.043505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:16.887002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f999322-3d18-4ae6-ad73-e286203d921e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:16.887884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f999322-3d18-4ae6-ad73-e286203d921e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:16.887884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f999322-3d18-4ae6-ad73-e286203d921e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:16.893084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 648/1307] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:16.899185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a1056-b51c-47e3-863b-eaac4b9e0dce None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:16.899185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a1056-b51c-47e3-863b-eaac4b9e0dce None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:16.899185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a1056-b51c-47e3-863b-eaac4b9e0dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:16.983625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-5e2a1056-b51c-47e3-863b-eaac4b9e0dce None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:16.985018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5e2a1056-b51c-47e3-863b-eaac4b9e0dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:17.115082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 660/1308] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:17.435431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ab944-14fa-49e9-a9b0-70dac46ca6ec None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:17.435732 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ab944-14fa-49e9-a9b0-70dac46ca6ec None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:17.436133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ab944-14fa-49e9-a9b0-70dac46ca6ec None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:17.438024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 649/1309] 10.222.0.84 () {54 vars in 965 bytes} [Wed Aug 24 11:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:17.444538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32462c2-967a-434e-8e09-22f6a4fba95d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:17.444993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32462c2-967a-434e-8e09-22f6a4fba95d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:17.445338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32462c2-967a-434e-8e09-22f6a4fba95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:17.526046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c32462c2-967a-434e-8e09-22f6a4fba95d None None] MFA Rules not processed for user `372e96bd3361434e9169b783f0c9ea13`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:17.527170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c32462c2-967a-434e-8e09-22f6a4fba95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:17.549036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:17.614968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 661/1310] 10.222.0.84 () {58 vars in 1062 bytes} [Wed Aug 24 11:23:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:17.624697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ab944-14fa-49e9-a9b0-70dac46ca6ec None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:17.626573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9d9ab944-14fa-49e9-a9b0-70dac46ca6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:17.771475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ab944-14fa-49e9-a9b0-70dac46ca6ec None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:17.862816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbcc12-1eef-459b-a312-cafbfb956c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:17.863819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbcc12-1eef-459b-a312-cafbfb956c79 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:17.864368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbcc12-1eef-459b-a312-cafbfb956c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:17.864879 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbcc12-1eef-459b-a312-cafbfb956c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:17.866681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7cbbcc12-1eef-459b-a312-cafbfb956c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:17.930103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:17.992129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbcc12-1eef-459b-a312-cafbfb956c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:17.993627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbcc12-1eef-459b-a312-cafbfb956c79 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:18.018146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 650/1311] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:20.278411 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21cc41-0aaa-4966-b2c1-75dc95780ff0 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:20.278971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21cc41-0aaa-4966-b2c1-75dc95780ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:20.278971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21cc41-0aaa-4966-b2c1-75dc95780ff0 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:20.281137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 662/1312] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:23:20.286600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209cdb1-1ef8-48f3-9a21-0d9a35c3e602 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:20.287135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209cdb1-1ef8-48f3-9a21-0d9a35c3e602 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:20.287216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209cdb1-1ef8-48f3-9a21-0d9a35c3e602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:20.381056 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0209cdb1-1ef8-48f3-9a21-0d9a35c3e602 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:20.382424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0209cdb1-1ef8-48f3-9a21-0d9a35c3e602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:20.490952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 651/1313] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:20.775388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21cc41-0aaa-4966-b2c1-75dc95780ff0 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:20.777010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ce21cc41-0aaa-4966-b2c1-75dc95780ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:20.882873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21cc41-0aaa-4966-b2c1-75dc95780ff0 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:20.958239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:20.960785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7038444-7148-4aea-9d27-3141024d48ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:20.961407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038444-7148-4aea-9d27-3141024d48ea service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:20.961679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038444-7148-4aea-9d27-3141024d48ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:20.961942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038444-7148-4aea-9d27-3141024d48ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:20.963511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f7038444-7148-4aea-9d27-3141024d48ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:21.074691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7038444-7148-4aea-9d27-3141024d48ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:21.076121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7038444-7148-4aea-9d27-3141024d48ea service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:21.094705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 663/1314] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:23.626159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420b7d4-2954-4cc4-bcb8-0e632c83d883 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:23.626704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420b7d4-2954-4cc4-bcb8-0e632c83d883 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:23.626704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420b7d4-2954-4cc4-bcb8-0e632c83d883 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:23.628074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 652/1315] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:23.634102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb755ae-cd00-4ee7-86b9-9f9bfba70db0 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:23.634538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb755ae-cd00-4ee7-86b9-9f9bfba70db0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:23.634635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb755ae-cd00-4ee7-86b9-9f9bfba70db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:23.700067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cbb755ae-cd00-4ee7-86b9-9f9bfba70db0 None None] MFA Rules not processed for user `417da11974af471a8afca3fc24034882`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:23.701310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cbb755ae-cd00-4ee7-86b9-9f9bfba70db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:23.756601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:23.800292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 664/1316] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:23:24.103342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71f924-751d-4873-a93a-daf01bda7b21 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:24.103620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71f924-751d-4873-a93a-daf01bda7b21 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:24.103891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71f924-751d-4873-a93a-daf01bda7b21 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:24.105277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 653/1317] 10.222.0.84 () {54 vars in 965 bytes} [Wed Aug 24 11:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:24.111273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a542d3b-ed1f-4acf-80ca-3fbde6fe21b7 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:24.111866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a542d3b-ed1f-4acf-80ca-3fbde6fe21b7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:24.112145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a542d3b-ed1f-4acf-80ca-3fbde6fe21b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:24.186714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-0a542d3b-ed1f-4acf-80ca-3fbde6fe21b7 None None] MFA Rules not processed for user `372e96bd3361434e9169b783f0c9ea13`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:24.187900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0a542d3b-ed1f-4acf-80ca-3fbde6fe21b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:24.262325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 665/1318] 10.222.0.84 () {58 vars in 1062 bytes} [Wed Aug 24 11:23:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:24.271551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71f924-751d-4873-a93a-daf01bda7b21 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:24.273090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ee71f924-751d-4873-a93a-daf01bda7b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:24.282747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:24.376005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71f924-751d-4873-a93a-daf01bda7b21 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:24.449624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e4438-1db8-463a-8958-eb60a99e6835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:24.450226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e4438-1db8-463a-8958-eb60a99e6835 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:24.450501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e4438-1db8-463a-8958-eb60a99e6835 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:24.450764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e4438-1db8-463a-8958-eb60a99e6835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:24.452352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1d7e4438-1db8-463a-8958-eb60a99e6835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:24.513532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:24.559421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e4438-1db8-463a-8958-eb60a99e6835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:24.560874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e4438-1db8-463a-8958-eb60a99e6835 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:24.578798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 654/1319] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:27.608105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89eb85-7ec1-4e46-b943-356f22fcd701 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:27.608678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89eb85-7ec1-4e46-b943-356f22fcd701 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:27.608678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89eb85-7ec1-4e46-b943-356f22fcd701 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:27.610041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 666/1320] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:27.615712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91347b5f-5e43-409c-995f-bf5788b60128 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:27.616208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91347b5f-5e43-409c-995f-bf5788b60128 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:27.616315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91347b5f-5e43-409c-995f-bf5788b60128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:27.681428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-91347b5f-5e43-409c-995f-bf5788b60128 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:27.682626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-91347b5f-5e43-409c-995f-bf5788b60128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:27.782017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 655/1321] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:28.077955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89eb85-7ec1-4e46-b943-356f22fcd701 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:28.079471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ef89eb85-7ec1-4e46-b943-356f22fcd701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:28.129685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:28.181491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89eb85-7ec1-4e46-b943-356f22fcd701 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:28.252036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0883e4d-ffb0-440a-bd45-a286c7fed16e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:28.252631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0883e4d-ffb0-440a-bd45-a286c7fed16e service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:28.252932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0883e4d-ffb0-440a-bd45-a286c7fed16e service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:28.253202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0883e4d-ffb0-440a-bd45-a286c7fed16e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:28.255049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c0883e4d-ffb0-440a-bd45-a286c7fed16e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:28.349155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:28.362170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0883e4d-ffb0-440a-bd45-a286c7fed16e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:28.364171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0883e4d-ffb0-440a-bd45-a286c7fed16e service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:28.381951 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 667/1322] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:31.712417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb6ed6-edd5-4ce0-b425-214b00641a7e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:31.713010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb6ed6-edd5-4ce0-b425-214b00641a7e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:31.713104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb6ed6-edd5-4ce0-b425-214b00641a7e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:31.714602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 656/1323] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:31.719703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93591bd1-7099-4609-945c-b904ede27b8d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:31.720104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93591bd1-7099-4609-945c-b904ede27b8d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:31.720243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93591bd1-7099-4609-945c-b904ede27b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:31.790574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-93591bd1-7099-4609-945c-b904ede27b8d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:31.791838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-93591bd1-7099-4609-945c-b904ede27b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:31.897614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 668/1324] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:32.182322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb6ed6-edd5-4ce0-b425-214b00641a7e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:32.184261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a6bb6ed6-edd5-4ce0-b425-214b00641a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:32.206563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:32.299658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb6ed6-edd5-4ce0-b425-214b00641a7e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:32.375839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81661c83-1adf-4d1f-b071-4d1d7b8c82c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:32.376448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661c83-1adf-4d1f-b071-4d1d7b8c82c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:32.376724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661c83-1adf-4d1f-b071-4d1d7b8c82c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:32.377023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661c83-1adf-4d1f-b071-4d1d7b8c82c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:32.378747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-81661c83-1adf-4d1f-b071-4d1d7b8c82c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:32.456120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:32.495644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81661c83-1adf-4d1f-b071-4d1d7b8c82c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:32.497125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81661c83-1adf-4d1f-b071-4d1d7b8c82c6 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:32.517554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 657/1325] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:34.580537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2826d-3060-4dd1-b0f2-a3a30aca1b33 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:34.581127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2826d-3060-4dd1-b0f2-a3a30aca1b33 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:34.581127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2826d-3060-4dd1-b0f2-a3a30aca1b33 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:34.582702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 669/1326] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:34.588891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffa216-c122-44eb-9ee9-6046cd2356b1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:34.589442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffa216-c122-44eb-9ee9-6046cd2356b1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:34.589442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffa216-c122-44eb-9ee9-6046cd2356b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:34.671873 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-adffa216-c122-44eb-9ee9-6046cd2356b1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:34.673125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-adffa216-c122-44eb-9ee9-6046cd2356b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:34.775342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 658/1327] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:35.061237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2826d-3060-4dd1-b0f2-a3a30aca1b33 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:35.062571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-74d2826d-3060-4dd1-b0f2-a3a30aca1b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:35.124040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:35.167634 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2826d-3060-4dd1-b0f2-a3a30aca1b33 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:35.244120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0bade-3e4e-4a52-8393-9e3ceb27a3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:35.244718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0bade-3e4e-4a52-8393-9e3ceb27a3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:35.245017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0bade-3e4e-4a52-8393-9e3ceb27a3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:35.245278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0bade-3e4e-4a52-8393-9e3ceb27a3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:35.246852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-98c0bade-3e4e-4a52-8393-9e3ceb27a3a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:35.363537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:35.365500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0bade-3e4e-4a52-8393-9e3ceb27a3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:35.366972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0bade-3e4e-4a52-8393-9e3ceb27a3a1 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:35.388342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 670/1328] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:37.443039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4819b05-df08-4ff0-b93d-947ce237ca45 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:37.443622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4819b05-df08-4ff0-b93d-947ce237ca45 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:37.443622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4819b05-df08-4ff0-b93d-947ce237ca45 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:37.444826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 659/1329] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:37.450169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a079f9-6e4a-41bf-bd5e-0e81d6b0ecce None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:37.450583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a079f9-6e4a-41bf-bd5e-0e81d6b0ecce None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:37.450740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a079f9-6e4a-41bf-bd5e-0e81d6b0ecce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:37.523452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b8a079f9-6e4a-41bf-bd5e-0e81d6b0ecce None None] MFA Rules not processed for user `417da11974af471a8afca3fc24034882`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:37.524693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b8a079f9-6e4a-41bf-bd5e-0e81d6b0ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:37.622789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 671/1330] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:37.922301 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4819b05-df08-4ff0-b93d-947ce237ca45 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:37.923915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e4819b05-df08-4ff0-b93d-947ce237ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:37.955080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:38.032201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4819b05-df08-4ff0-b93d-947ce237ca45 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:38.107609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33a4ef-09c5-4e7a-b6a9-2f59eaaddc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:38.108207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33a4ef-09c5-4e7a-b6a9-2f59eaaddc43 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:38.108483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33a4ef-09c5-4e7a-b6a9-2f59eaaddc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:38.108750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33a4ef-09c5-4e7a-b6a9-2f59eaaddc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:38.110351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9a33a4ef-09c5-4e7a-b6a9-2f59eaaddc43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:38.187041 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:38.219676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33a4ef-09c5-4e7a-b6a9-2f59eaaddc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:38.221157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33a4ef-09c5-4e7a-b6a9-2f59eaaddc43 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:38.239797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 660/1331] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:40.929719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd7a01-260a-4531-9148-fe6d01195c95 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:40.930230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd7a01-260a-4531-9148-fe6d01195c95 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:40.930230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd7a01-260a-4531-9148-fe6d01195c95 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:40.931568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 672/1332] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:40.937051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b43ee-531e-4a17-b379-dc135bf9cdc3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:40.937477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b43ee-531e-4a17-b379-dc135bf9cdc3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:40.937573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b43ee-531e-4a17-b379-dc135bf9cdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:41.003005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-066b43ee-531e-4a17-b379-dc135bf9cdc3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:41.004317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-066b43ee-531e-4a17-b379-dc135bf9cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:41.098885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 661/1333] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:41.382521 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd7a01-260a-4531-9148-fe6d01195c95 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:41.384056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-70fd7a01-260a-4531-9148-fe6d01195c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:41.455253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:41.489804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd7a01-260a-4531-9148-fe6d01195c95 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:41.563740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fae532-0672-4b70-ade3-21c39ce89051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:41.564340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fae532-0672-4b70-ade3-21c39ce89051 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:41.564616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fae532-0672-4b70-ade3-21c39ce89051 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:41.564918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fae532-0672-4b70-ade3-21c39ce89051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:41.566770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-62fae532-0672-4b70-ade3-21c39ce89051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:41.677335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fae532-0672-4b70-ade3-21c39ce89051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:41.678777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fae532-0672-4b70-ade3-21c39ce89051 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:41.688596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:41.697500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 673/1334] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:43.659202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a625c7-ea2d-489f-bdc5-5542edb25e8c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:43.659764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a625c7-ea2d-489f-bdc5-5542edb25e8c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:43.659861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a625c7-ea2d-489f-bdc5-5542edb25e8c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:43.661718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 662/1335] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:43.667198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9964b-d3a1-4e11-b39b-aba8f4ca58f8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:43.667653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9964b-d3a1-4e11-b39b-aba8f4ca58f8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:43.667748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9964b-d3a1-4e11-b39b-aba8f4ca58f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:43.734592 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-24f9964b-d3a1-4e11-b39b-aba8f4ca58f8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:43.735781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-24f9964b-d3a1-4e11-b39b-aba8f4ca58f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:43.830896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 674/1336] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:44.126241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a625c7-ea2d-489f-bdc5-5542edb25e8c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:44.129067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f6a625c7-ea2d-489f-bdc5-5542edb25e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:44.169277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:44.234008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a625c7-ea2d-489f-bdc5-5542edb25e8c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:44.307796 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfcbd4-e1ee-445b-8a54-c7087b67b3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:44.308405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfcbd4-e1ee-445b-8a54-c7087b67b3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:44.308683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfcbd4-e1ee-445b-8a54-c7087b67b3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:44.308984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfcbd4-e1ee-445b-8a54-c7087b67b3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:44.310689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9acfcbd4-e1ee-445b-8a54-c7087b67b3f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:44.394963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:44.419855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfcbd4-e1ee-445b-8a54-c7087b67b3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:44.421335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfcbd4-e1ee-445b-8a54-c7087b67b3f5 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:44.442186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 663/1337] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:47.350429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84957990-0657-4dd8-a246-affc0125e8e4 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:47.351017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84957990-0657-4dd8-a246-affc0125e8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:47.351017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84957990-0657-4dd8-a246-affc0125e8e4 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:47.352321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 675/1338] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:47.357802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d54c12-bb88-42c0-9e57-090b65460048 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:47.358204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d54c12-bb88-42c0-9e57-090b65460048 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:47.358346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d54c12-bb88-42c0-9e57-090b65460048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:47.427046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-92d54c12-bb88-42c0-9e57-090b65460048 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:47.428311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-92d54c12-bb88-42c0-9e57-090b65460048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:47.540563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 664/1339] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:47.821506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84957990-0657-4dd8-a246-affc0125e8e4 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:47.823042 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-84957990-0657-4dd8-a246-affc0125e8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:47.898945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:47.925721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84957990-0657-4dd8-a246-affc0125e8e4 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:48.001888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cb5a2-bfbb-4aa7-8676-bb7c4e24f305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:48.002497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cb5a2-bfbb-4aa7-8676-bb7c4e24f305 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:48.002773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cb5a2-bfbb-4aa7-8676-bb7c4e24f305 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:48.003578 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cb5a2-bfbb-4aa7-8676-bb7c4e24f305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:48.006076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-747cb5a2-bfbb-4aa7-8676-bb7c4e24f305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:48.118138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747cb5a2-bfbb-4aa7-8676-bb7c4e24f305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:48.119578 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747cb5a2-bfbb-4aa7-8676-bb7c4e24f305 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:48.138765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 676/1340] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:52.791061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316792d7-323a-4526-b133-47f6b0718397 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:52.791663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316792d7-323a-4526-b133-47f6b0718397 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:52.791663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316792d7-323a-4526-b133-47f6b0718397 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:52.793049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 665/1341] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:52.798471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02544a-a903-41bd-a23c-f216aeeb04b9 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:52.798931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02544a-a903-41bd-a23c-f216aeeb04b9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:52.799020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02544a-a903-41bd-a23c-f216aeeb04b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:52.805039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:52.867800 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-1a02544a-a903-41bd-a23c-f216aeeb04b9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:52.869062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1a02544a-a903-41bd-a23c-f216aeeb04b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:52.968209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 677/1342] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:53.255905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316792d7-323a-4526-b133-47f6b0718397 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:53.257528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-316792d7-323a-4526-b133-47f6b0718397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:53.312282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:53.366968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316792d7-323a-4526-b133-47f6b0718397 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:53.442364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234c72-431b-441e-86ae-803da7af5ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:53.442980 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234c72-431b-441e-86ae-803da7af5ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:53.443254 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234c72-431b-441e-86ae-803da7af5ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:53.443521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234c72-431b-441e-86ae-803da7af5ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:53.445156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d5234c72-431b-441e-86ae-803da7af5ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:53.541591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:53.559687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234c72-431b-441e-86ae-803da7af5ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:53.561196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234c72-431b-441e-86ae-803da7af5ec3 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:53.580200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 666/1343] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:23:56.756619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6679526-ee13-4f2b-929b-e68d283d9186 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:56.757204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6679526-ee13-4f2b-929b-e68d283d9186 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:56.757204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6679526-ee13-4f2b-929b-e68d283d9186 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:56.758770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 678/1344] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:23:56.763972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aef27a-4d1a-4a22-93cd-9b9d8ad0e8f6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:56.764418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aef27a-4d1a-4a22-93cd-9b9d8ad0e8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:56.764526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aef27a-4d1a-4a22-93cd-9b9d8ad0e8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:56.831521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-31aef27a-4d1a-4a22-93cd-9b9d8ad0e8f6 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:23:56.832733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-31aef27a-4d1a-4a22-93cd-9b9d8ad0e8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:56.929158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 667/1345] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:23:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:23:57.211846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6679526-ee13-4f2b-929b-e68d283d9186 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:23:57.213412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d6679526-ee13-4f2b-929b-e68d283d9186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:57.305746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:23:57.322662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6679526-ee13-4f2b-929b-e68d283d9186 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:23:57.403897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4def47-e6d6-4d9f-951b-84ed3919f533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:23:57.404517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4def47-e6d6-4d9f-951b-84ed3919f533 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:23:57.404857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4def47-e6d6-4d9f-951b-84ed3919f533 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:23:57.405113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4def47-e6d6-4d9f-951b-84ed3919f533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:23:57.406798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5f4def47-e6d6-4d9f-951b-84ed3919f533 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:23:57.520105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4def47-e6d6-4d9f-951b-84ed3919f533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:23:57.521526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4def47-e6d6-4d9f-951b-84ed3919f533 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:23:57.541224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 679/1346] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:23:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:02.065921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9aa25-f739-4460-b4b8-9be78095e290 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:02.066448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9aa25-f739-4460-b4b8-9be78095e290 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:02.066543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9aa25-f739-4460-b4b8-9be78095e290 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:02.067900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 668/1347] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:02.073549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4c2d7-eaca-4b8e-bfd4-48f9626d0ac1 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:02.074040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4c2d7-eaca-4b8e-bfd4-48f9626d0ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:02.074134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4c2d7-eaca-4b8e-bfd4-48f9626d0ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:02.090504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:02.147033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b4f4c2d7-eaca-4b8e-bfd4-48f9626d0ac1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:02.148339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b4f4c2d7-eaca-4b8e-bfd4-48f9626d0ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:02.260575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 680/1348] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:02.551593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9aa25-f739-4460-b4b8-9be78095e290 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:02.553176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f2a9aa25-f739-4460-b4b8-9be78095e290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:02.614884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:02.670571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9aa25-f739-4460-b4b8-9be78095e290 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:02.748825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef821d-92cb-4ee0-a753-b0fae38795a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:02.749449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef821d-92cb-4ee0-a753-b0fae38795a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:02.749722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef821d-92cb-4ee0-a753-b0fae38795a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:02.749989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef821d-92cb-4ee0-a753-b0fae38795a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:02.751589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b4ef821d-92cb-4ee0-a753-b0fae38795a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:02.864707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:02.868200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef821d-92cb-4ee0-a753-b0fae38795a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:02.869677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef821d-92cb-4ee0-a753-b0fae38795a1 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:02.888754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 669/1349] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:07.926765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee090e32-0b14-434a-ab06-0d1bb0f8fc77 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:07.927256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee090e32-0b14-434a-ab06-0d1bb0f8fc77 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:07.927387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee090e32-0b14-434a-ab06-0d1bb0f8fc77 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:07.930001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 681/1350] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:24:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:07.946104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e125e1-5375-4b91-b803-fdddbd637fb5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:07.946724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e125e1-5375-4b91-b803-fdddbd637fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:07.946724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e125e1-5375-4b91-b803-fdddbd637fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:08.110928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b7e125e1-5375-4b91-b803-fdddbd637fb5 None None] MFA Rules not processed for user `c7cc35f5ddf0446e90ded5c85f21f37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:08.112669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b7e125e1-5375-4b91-b803-fdddbd637fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:08.242995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 670/1351] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:12.258729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=59 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:12.416044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf0eb-6b33-4b92-abe8-d62d040ecb28 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:12.416448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf0eb-6b33-4b92-abe8-d62d040ecb28 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:12.416582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbf0eb-6b33-4b92-abe8-d62d040ecb28 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:12.417965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 682/1352] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:12.423400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e433f48-ad7b-4f29-b3f9-166f557373e0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:12.424861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e433f48-ad7b-4f29-b3f9-166f557373e0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:12.425320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e433f48-ad7b-4f29-b3f9-166f557373e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:12.482406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:12.506668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5e433f48-ad7b-4f29-b3f9-166f557373e0 None None] MFA Rules not processed for user `c7cc35f5ddf0446e90ded5c85f21f37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:12.507859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5e433f48-ad7b-4f29-b3f9-166f557373e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:12.623398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 671/1353] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:24:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:13.911461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00062b5-358c-4d61-b976-50f3bf250b47 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:13.911983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00062b5-358c-4d61-b976-50f3bf250b47 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:13.911983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00062b5-358c-4d61-b976-50f3bf250b47 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:13.913303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 683/1354] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:13.918776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e5361-9c7c-45e4-b458-c2fa9cfdf581 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:13.919236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e5361-9c7c-45e4-b458-c2fa9cfdf581 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:13.919335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e5361-9c7c-45e4-b458-c2fa9cfdf581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:13.998283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f19e5361-9c7c-45e4-b458-c2fa9cfdf581 None None] MFA Rules not processed for user `c7cc35f5ddf0446e90ded5c85f21f37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:13.999476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f19e5361-9c7c-45e4-b458-c2fa9cfdf581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:14.042598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:14.109234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 672/1355] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:24:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:17.657383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59236fca-211a-4645-94a6-02dd416ef2fb None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:17.657861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59236fca-211a-4645-94a6-02dd416ef2fb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:17.658000 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59236fca-211a-4645-94a6-02dd416ef2fb None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:17.660098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 684/1356] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:17.666383 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097a6f5-6f7d-4147-95e8-152ee79d7ecb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:17.666804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097a6f5-6f7d-4147-95e8-152ee79d7ecb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:17.666913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097a6f5-6f7d-4147-95e8-152ee79d7ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:17.756634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6097a6f5-6f7d-4147-95e8-152ee79d7ecb None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:17.757864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6097a6f5-6f7d-4147-95e8-152ee79d7ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:17.894395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 673/1357] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:18.207077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10305b-801d-43f9-8585-7621ffc0702b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:18.207634 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10305b-801d-43f9-8585-7621ffc0702b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:18.208135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10305b-801d-43f9-8585-7621ffc0702b None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:18.210615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 685/1358] 10.222.0.84 () {54 vars in 962 bytes} [Wed Aug 24 11:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:18.221224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae3248-90ea-4539-b139-f87fe6f2dfa4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:18.221642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae3248-90ea-4539-b139-f87fe6f2dfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:18.221905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae3248-90ea-4539-b139-f87fe6f2dfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:18.239537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:18.326157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4dae3248-90ea-4539-b139-f87fe6f2dfa4 None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:18.327536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4dae3248-90ea-4539-b139-f87fe6f2dfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:18.454518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 674/1359] 10.222.0.84 () {58 vars in 1059 bytes} [Wed Aug 24 11:24:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:18.463729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10305b-801d-43f9-8585-7621ffc0702b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:18.465398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9d10305b-801d-43f9-8585-7621ffc0702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:18.619324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10305b-801d-43f9-8585-7621ffc0702b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:18.706906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:18.751324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d444b2f-2954-42a9-b6aa-7270a9462a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:18.752279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d444b2f-2954-42a9-b6aa-7270a9462a73 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:18.752622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d444b2f-2954-42a9-b6aa-7270a9462a73 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:18.752997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d444b2f-2954-42a9-b6aa-7270a9462a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:18.754654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3d444b2f-2954-42a9-b6aa-7270a9462a73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:18.849861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d444b2f-2954-42a9-b6aa-7270a9462a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:18.851258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d444b2f-2954-42a9-b6aa-7270a9462a73 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:18.870604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 686/1360] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:21.054023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a63da5-a9dd-4897-9564-1f1e290d16b1 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:21.054556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a63da5-a9dd-4897-9564-1f1e290d16b1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:21.054556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a63da5-a9dd-4897-9564-1f1e290d16b1 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:21.056044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 675/1361] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:21.061372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971ef1-390e-4898-9d13-05e37a6df1fb None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:21.061850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971ef1-390e-4898-9d13-05e37a6df1fb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:21.061938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971ef1-390e-4898-9d13-05e37a6df1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:21.133412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-8e971ef1-390e-4898-9d13-05e37a6df1fb None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:21.134588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8e971ef1-390e-4898-9d13-05e37a6df1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:21.173374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:21.244567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 687/1362] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:21.539362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a63da5-a9dd-4897-9564-1f1e290d16b1 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:21.540869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a8a63da5-a9dd-4897-9564-1f1e290d16b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:21.618507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:21.711158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a63da5-a9dd-4897-9564-1f1e290d16b1 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:21.837935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7f7d1-38a6-4682-a554-4538e96968ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:21.838700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7f7d1-38a6-4682-a554-4538e96968ce service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:21.839083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7f7d1-38a6-4682-a554-4538e96968ce service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:21.839458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7f7d1-38a6-4682-a554-4538e96968ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:21.841449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-92d7f7d1-38a6-4682-a554-4538e96968ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:21.867595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:21.959911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7f7d1-38a6-4682-a554-4538e96968ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:21.961755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7f7d1-38a6-4682-a554-4538e96968ce service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:21.985564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 676/1363] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:24.458766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c871601-1240-4a58-9171-4ef048fc8259 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:24.459650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c871601-1240-4a58-9171-4ef048fc8259 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:24.460075 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c871601-1240-4a58-9171-4ef048fc8259 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:24.461653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 688/1364] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:24.466974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1662f-2911-41bd-b2ca-5f5c94fc2976 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:24.467743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1662f-2911-41bd-b2ca-5f5c94fc2976 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:24.468202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1662f-2911-41bd-b2ca-5f5c94fc2976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:24.538053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a2d1662f-2911-41bd-b2ca-5f5c94fc2976 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:24.539254 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a2d1662f-2911-41bd-b2ca-5f5c94fc2976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:24.644287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:24.648960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 677/1365] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:24.937553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a5fe8-3e94-4a04-83e0-82acd9923aa6 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:24.938330 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a5fe8-3e94-4a04-83e0-82acd9923aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:24.938624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a5fe8-3e94-4a04-83e0-82acd9923aa6 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:24.940609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 689/1366] 10.222.0.84 () {54 vars in 962 bytes} [Wed Aug 24 11:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:24.946312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745a8a2-1fe3-4c7e-97be-5d34e157c20f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:24.946672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745a8a2-1fe3-4c7e-97be-5d34e157c20f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:24.947024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745a8a2-1fe3-4c7e-97be-5d34e157c20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:25.027967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7745a8a2-1fe3-4c7e-97be-5d34e157c20f None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:25.029326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7745a8a2-1fe3-4c7e-97be-5d34e157c20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:25.157131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 678/1367] 10.222.0.84 () {58 vars in 1059 bytes} [Wed Aug 24 11:24:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:25.165433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a5fe8-3e94-4a04-83e0-82acd9923aa6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:25.166924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9b3a5fe8-3e94-4a04-83e0-82acd9923aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:25.329889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a5fe8-3e94-4a04-83e0-82acd9923aa6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:25.351851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:25.468279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d70db-c914-4032-9c8e-63133f5fdf87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:25.469106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d70db-c914-4032-9c8e-63133f5fdf87 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:25.469202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d70db-c914-4032-9c8e-63133f5fdf87 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:25.469427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d70db-c914-4032-9c8e-63133f5fdf87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:25.471692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-235d70db-c914-4032-9c8e-63133f5fdf87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:25.563470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d70db-c914-4032-9c8e-63133f5fdf87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:25.565127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d70db-c914-4032-9c8e-63133f5fdf87 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:25.584395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 690/1368] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:26.027692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92fed8-1bd3-497d-bb3b-4dc5f5c0771c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:26.028290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92fed8-1bd3-497d-bb3b-4dc5f5c0771c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:26.028604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92fed8-1bd3-497d-bb3b-4dc5f5c0771c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:26.032172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 679/1369] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:26.036262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880f136-f61b-44be-8bbd-b3bb84139ed8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:26.036575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880f136-f61b-44be-8bbd-b3bb84139ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:26.036856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880f136-f61b-44be-8bbd-b3bb84139ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:26.073221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:26.112350 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e880f136-f61b-44be-8bbd-b3bb84139ed8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:26.113652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e880f136-f61b-44be-8bbd-b3bb84139ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:26.216933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 691/1370] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:26.807461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dba7f-1cbe-4a47-9bc0-da1d232a5bc1 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:26.807461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dba7f-1cbe-4a47-9bc0-da1d232a5bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:26.807461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dba7f-1cbe-4a47-9bc0-da1d232a5bc1 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:26.809340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 680/1371] 10.222.0.84 () {54 vars in 964 bytes} [Wed Aug 24 11:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:26.813037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d720e-3458-427d-a0a6-89a498c27d76 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:26.813538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d720e-3458-427d-a0a6-89a498c27d76 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:26.813694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d720e-3458-427d-a0a6-89a498c27d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:26.894773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-139d720e-3458-427d-a0a6-89a498c27d76 None None] MFA Rules not processed for user `d5ce16d5ea8240debfcb69ac7b1f71c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:26.895983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-139d720e-3458-427d-a0a6-89a498c27d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:26.946756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:26.989520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 692/1372] 10.222.0.84 () {58 vars in 1061 bytes} [Wed Aug 24 11:24:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:26.998574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dba7f-1cbe-4a47-9bc0-da1d232a5bc1 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:27.000426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1a9dba7f-1cbe-4a47-9bc0-da1d232a5bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:27.062911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:27.119743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dba7f-1cbe-4a47-9bc0-da1d232a5bc1 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:27.198285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00bf62-2daa-4ba8-9420-1d05a4d114da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:27.198901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00bf62-2daa-4ba8-9420-1d05a4d114da service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:27.199179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00bf62-2daa-4ba8-9420-1d05a4d114da service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:27.199447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00bf62-2daa-4ba8-9420-1d05a4d114da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:27.201135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7b00bf62-2daa-4ba8-9420-1d05a4d114da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:27.303020 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:27.316995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00bf62-2daa-4ba8-9420-1d05a4d114da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:27.318532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00bf62-2daa-4ba8-9420-1d05a4d114da service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:27.350342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 681/1373] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:27.834028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d119e1-9877-435d-bdcd-fb0d703dc930 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:27.834563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d119e1-9877-435d-bdcd-fb0d703dc930 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:27.834563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d119e1-9877-435d-bdcd-fb0d703dc930 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:27.835900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 693/1374] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:27.841194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e7c3b-482a-40e6-bd5f-21fc2ef13a1c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:27.841478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e7c3b-482a-40e6-bd5f-21fc2ef13a1c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:27.841779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e7c3b-482a-40e6-bd5f-21fc2ef13a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:27.911968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-615e7c3b-482a-40e6-bd5f-21fc2ef13a1c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:27.913213 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-615e7c3b-482a-40e6-bd5f-21fc2ef13a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:28.016752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 682/1375] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:28.313039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d119e1-9877-435d-bdcd-fb0d703dc930 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:28.315222 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-95d119e1-9877-435d-bdcd-fb0d703dc930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:28.461757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d119e1-9877-435d-bdcd-fb0d703dc930 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:28.523641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:28.586166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9289ccb-b0d0-4772-8d51-c59e862176fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:28.586769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9289ccb-b0d0-4772-8d51-c59e862176fb service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:28.587043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9289ccb-b0d0-4772-8d51-c59e862176fb service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:28.587307 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9289ccb-b0d0-4772-8d51-c59e862176fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:28.588932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e9289ccb-b0d0-4772-8d51-c59e862176fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:28.685232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9289ccb-b0d0-4772-8d51-c59e862176fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:28.686830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9289ccb-b0d0-4772-8d51-c59e862176fb service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:28.705582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 694/1376] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:30.819797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69472264-f807-4345-a384-b9273559b944 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:30.820355 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69472264-f807-4345-a384-b9273559b944 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:30.820355 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69472264-f807-4345-a384-b9273559b944 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:30.821747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 683/1377] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:30.827254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ef5cb-1d8c-436a-a0ef-46fa2a4f20ee None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:30.827642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ef5cb-1d8c-436a-a0ef-46fa2a4f20ee None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:30.827797 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ef5cb-1d8c-436a-a0ef-46fa2a4f20ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:30.904757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-357ef5cb-1d8c-436a-a0ef-46fa2a4f20ee None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:30.906007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-357ef5cb-1d8c-436a-a0ef-46fa2a4f20ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:30.915091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:31.029600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 695/1378] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:31.357519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69472264-f807-4345-a384-b9273559b944 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:31.359371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-69472264-f807-4345-a384-b9273559b944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:31.426826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:31.519608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69472264-f807-4345-a384-b9273559b944 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:31.639340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6029e2a-636a-41b1-b989-fa515c4ea58a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:31.640014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6029e2a-636a-41b1-b989-fa515c4ea58a service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:31.640396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6029e2a-636a-41b1-b989-fa515c4ea58a service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:31.640771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6029e2a-636a-41b1-b989-fa515c4ea58a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:31.642815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f6029e2a-636a-41b1-b989-fa515c4ea58a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:31.664853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:31.742402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6029e2a-636a-41b1-b989-fa515c4ea58a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:31.744179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6029e2a-636a-41b1-b989-fa515c4ea58a service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:31.764589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 684/1379] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:33.888171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd07d77-ad72-4125-9067-acdeda4ff5d0 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:33.889101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd07d77-ad72-4125-9067-acdeda4ff5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:33.889551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd07d77-ad72-4125-9067-acdeda4ff5d0 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:33.897125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f952-9b67-4416-927d-2c1afb4b8b0e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:33.897672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f952-9b67-4416-927d-2c1afb4b8b0e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:33.897672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f952-9b67-4416-927d-2c1afb4b8b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:33.900686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 696/1380] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:33.968867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-3e54f952-9b67-4416-927d-2c1afb4b8b0e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:33.970150 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3e54f952-9b67-4416-927d-2c1afb4b8b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:34.084984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:34.091222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 685/1381] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:34.385743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd07d77-ad72-4125-9067-acdeda4ff5d0 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:34.387274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7fd07d77-ad72-4125-9067-acdeda4ff5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:34.537376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:34.548274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd07d77-ad72-4125-9067-acdeda4ff5d0 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:34.683404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82816765-018b-4fc2-b52f-83675aeb53b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:34.684013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82816765-018b-4fc2-b52f-83675aeb53b8 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:34.684290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82816765-018b-4fc2-b52f-83675aeb53b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:34.684618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82816765-018b-4fc2-b52f-83675aeb53b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:34.686408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-82816765-018b-4fc2-b52f-83675aeb53b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:34.789183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82816765-018b-4fc2-b52f-83675aeb53b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:34.792083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82816765-018b-4fc2-b52f-83675aeb53b8 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:34.805065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:34.813732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 697/1382] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:37.009557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137be9b-6f9b-4a32-b858-7fe0a939f51b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.010143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137be9b-6f9b-4a32-b858-7fe0a939f51b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.010143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137be9b-6f9b-4a32-b858-7fe0a939f51b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.011366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 686/1383] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:37.017712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb4e3c-7e4d-4ab4-837a-99ca12eb1df6 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.018188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb4e3c-7e4d-4ab4-837a-99ca12eb1df6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.018302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb4e3c-7e4d-4ab4-837a-99ca12eb1df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.090255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d5bb4e3c-7e4d-4ab4-837a-99ca12eb1df6 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:37.091481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d5bb4e3c-7e4d-4ab4-837a-99ca12eb1df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.206676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 698/1384] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:37.534932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137be9b-6f9b-4a32-b858-7fe0a939f51b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:37.536548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6137be9b-6f9b-4a32-b858-7fe0a939f51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.694427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137be9b-6f9b-4a32-b858-7fe0a939f51b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:37.769272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:37.816761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075a0f1-69e6-4d67-9391-790418b52aa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:37.817387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075a0f1-69e6-4d67-9391-790418b52aa5 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:37.817661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075a0f1-69e6-4d67-9391-790418b52aa5 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:37.817931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075a0f1-69e6-4d67-9391-790418b52aa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:37.819523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b075a0f1-69e6-4d67-9391-790418b52aa5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:37.933973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075a0f1-69e6-4d67-9391-790418b52aa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:37.935999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075a0f1-69e6-4d67-9391-790418b52aa5 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:37.960321 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 687/1385] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:40.307288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19fb25-cbeb-44c2-a720-67f32f912115 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.307885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19fb25-cbeb-44c2-a720-67f32f912115 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.307885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19fb25-cbeb-44c2-a720-67f32f912115 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.309315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 699/1386] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:40.316641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdd0b8-56a3-467f-9ec0-6d2d65bdaf57 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:40.317330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdd0b8-56a3-467f-9ec0-6d2d65bdaf57 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:40.317489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdd0b8-56a3-467f-9ec0-6d2d65bdaf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:40.393756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8bbdd0b8-56a3-467f-9ec0-6d2d65bdaf57 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:40.394969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8bbdd0b8-56a3-467f-9ec0-6d2d65bdaf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:40.430765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:40.499279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 688/1387] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:24:40.800423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19fb25-cbeb-44c2-a720-67f32f912115 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:40.802183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3c19fb25-cbeb-44c2-a720-67f32f912115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:40.887419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:40.962520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19fb25-cbeb-44c2-a720-67f32f912115 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:41.082995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289fc40-eedd-4715-bedb-92060e514894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:41.083640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289fc40-eedd-4715-bedb-92060e514894 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:41.083915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289fc40-eedd-4715-bedb-92060e514894 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:41.084185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289fc40-eedd-4715-bedb-92060e514894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:41.085891 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5289fc40-eedd-4715-bedb-92060e514894 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:41.126328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:41.179748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289fc40-eedd-4715-bedb-92060e514894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:41.181426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289fc40-eedd-4715-bedb-92060e514894 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:41.199579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 700/1388] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:42.911242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3c274-f62a-4ab4-be4e-8309da5d9319 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:42.911781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3c274-f62a-4ab4-be4e-8309da5d9319 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:42.911781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3c274-f62a-4ab4-be4e-8309da5d9319 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:42.913080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 689/1389] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:42.918351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd941ba-516e-4738-bd26-a85721438211 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:42.918803 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd941ba-516e-4738-bd26-a85721438211 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:42.918971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd941ba-516e-4738-bd26-a85721438211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:42.986429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-8bd941ba-516e-4738-bd26-a85721438211 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:42.987908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8bd941ba-516e-4738-bd26-a85721438211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:43.087225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 701/1390] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:43.305411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7919857-cba5-421c-8fc1-0719c0770f2b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:43.305620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7919857-cba5-421c-8fc1-0719c0770f2b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:43.305879 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7919857-cba5-421c-8fc1-0719c0770f2b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:43.307227 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 690/1391] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:43.312644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c85ff-f876-4919-9dea-fecf82da4576 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:43.312960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c85ff-f876-4919-9dea-fecf82da4576 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:43.313231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c85ff-f876-4919-9dea-fecf82da4576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:43.329018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:43.380876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-fd1c85ff-f876-4919-9dea-fecf82da4576 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:43.382085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fd1c85ff-f876-4919-9dea-fecf82da4576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:43.391034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a343a-6b19-4832-ab9e-14d0f2590b4b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:43.391661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a343a-6b19-4832-ab9e-14d0f2590b4b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:43.391794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a343a-6b19-4832-ab9e-14d0f2590b4b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:43.393090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 691/1392] 10.222.0.84 () {54 vars in 967 bytes} [Wed Aug 24 11:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:43.397795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd917da-62fb-4d08-a299-63e018f1a6f8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:43.398074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd917da-62fb-4d08-a299-63e018f1a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:43.398349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd917da-62fb-4d08-a299-63e018f1a6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:43.478616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8bd917da-62fb-4d08-a299-63e018f1a6f8 None None] MFA Rules not processed for user `75ac781cf2d84d95a789724e74e12acf`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:43.479929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8bd917da-62fb-4d08-a299-63e018f1a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:43.481040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 702/1393] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:43.557842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 692/1394] 10.222.0.84 () {58 vars in 1064 bytes} [Wed Aug 24 11:24:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:24:43.565695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c85ff-f876-4919-9dea-fecf82da4576 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:43.566877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fd1c85ff-f876-4919-9dea-fecf82da4576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:43.660040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c85ff-f876-4919-9dea-fecf82da4576 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:43.667838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:43.734998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936dd1f-34eb-4b43-9ac3-15504b687701 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ac781cf2d84d95a789724e74e12acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:43.735602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936dd1f-34eb-4b43-9ac3-15504b687701 service glance] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:43.736025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936dd1f-34eb-4b43-9ac3-15504b687701 service glance] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:43.736294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936dd1f-34eb-4b43-9ac3-15504b687701 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:43.738208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a936dd1f-34eb-4b43-9ac3-15504b687701 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:43.773097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd917da-62fb-4d08-a299-63e018f1a6f8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:43.774287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8bd917da-62fb-4d08-a299-63e018f1a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:43.803962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:43.850758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936dd1f-34eb-4b43-9ac3-15504b687701 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:43.852110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936dd1f-34eb-4b43-9ac3-15504b687701 service glance] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:43.871432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 703/1395] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:43.900609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd917da-62fb-4d08-a299-63e018f1a6f8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:44.021554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9f72e-bb2d-42f5-927d-e10738975af1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:44.022193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9f72e-bb2d-42f5-927d-e10738975af1 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:44.022430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9f72e-bb2d-42f5-927d-e10738975af1 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:44.022698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9f72e-bb2d-42f5-927d-e10738975af1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:44.024413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0ae9f72e-bb2d-42f5-927d-e10738975af1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:44.043504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:44.123799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9f72e-bb2d-42f5-927d-e10738975af1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:44.125554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9f72e-bb2d-42f5-927d-e10738975af1 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:44.146776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 693/1396] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:24:44.802636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732319e9-db6a-4b39-8cdf-e2ea9db33a00 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:44.803129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732319e9-db6a-4b39-8cdf-e2ea9db33a00 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:44.803221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732319e9-db6a-4b39-8cdf-e2ea9db33a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:44.848316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:44.869807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-732319e9-db6a-4b39-8cdf-e2ea9db33a00 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:44.871009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-732319e9-db6a-4b39-8cdf-e2ea9db33a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:44.958353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 704/1397] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:24:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:45.000216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c27c67-f8ed-4086-83e9-5e13546e3a82 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:45.000500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c27c67-f8ed-4086-83e9-5e13546e3a82 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:45.001341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c27c67-f8ed-4086-83e9-5e13546e3a82 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:45.002688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 694/1398] 10.222.0.84 () {54 vars in 943 bytes} [Wed Aug 24 11:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:45.009191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d684a-54d4-439b-82d1-eeee6814f608 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:45.009480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d684a-54d4-439b-82d1-eeee6814f608 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:45.009753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d684a-54d4-439b-82d1-eeee6814f608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:45.090987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-2b4d684a-54d4-439b-82d1-eeee6814f608 None None] MFA Rules not processed for user `1ba54c2312b348b08e0faae745747e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:45.092687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2b4d684a-54d4-439b-82d1-eeee6814f608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:45.169579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:45.216768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 705/1399] 10.222.0.84 () {58 vars in 1040 bytes} [Wed Aug 24 11:24:45 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:24:45.226510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c27c67-f8ed-4086-83e9-5e13546e3a82 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:45.228017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-13c27c67-f8ed-4086-83e9-5e13546e3a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:45.370546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c27c67-f8ed-4086-83e9-5e13546e3a82 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:45.381187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:45.489856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb0fb-5033-49e2-8422-4ca00e1c78da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:45.490468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb0fb-5033-49e2-8422-4ca00e1c78da service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:45.490741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb0fb-5033-49e2-8422-4ca00e1c78da service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:45.491010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb0fb-5033-49e2-8422-4ca00e1c78da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:45.492639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-db4bb0fb-5033-49e2-8422-4ca00e1c78da service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:45.576581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bb0fb-5033-49e2-8422-4ca00e1c78da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:45.578203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bb0fb-5033-49e2-8422-4ca00e1c78da service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:45.586522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 695/1400] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:24:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:24:46.379909 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f6c7b-8471-44d1-b9f6-57ce3b2e5cfa None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:46.380469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f6c7b-8471-44d1-b9f6-57ce3b2e5cfa None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:46.380881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f6c7b-8471-44d1-b9f6-57ce3b2e5cfa None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:46.382831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 706/1401] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:46.388178 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c81ff-5bed-49da-8496-ab562dd8b09e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:46.388968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c81ff-5bed-49da-8496-ab562dd8b09e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:46.389247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c81ff-5bed-49da-8496-ab562dd8b09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:46.439826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:46.461928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d69c81ff-5bed-49da-8496-ab562dd8b09e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:46.461928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d69c81ff-5bed-49da-8496-ab562dd8b09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:46.577095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 696/1402] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:46.881809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f6c7b-8471-44d1-b9f6-57ce3b2e5cfa None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:46.883316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1f1f6c7b-8471-44d1-b9f6-57ce3b2e5cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:47.063966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f6c7b-8471-44d1-b9f6-57ce3b2e5cfa None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:47.114293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:47.202137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea47c2-42a3-4c85-a230-b0e290d0f921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:47.202739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea47c2-42a3-4c85-a230-b0e290d0f921 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:47.203072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea47c2-42a3-4c85-a230-b0e290d0f921 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:47.203343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea47c2-42a3-4c85-a230-b0e290d0f921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:47.205486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b7ea47c2-42a3-4c85-a230-b0e290d0f921 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:47.311401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea47c2-42a3-4c85-a230-b0e290d0f921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:47.313051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ea47c2-42a3-4c85-a230-b0e290d0f921 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:47.337248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 707/1403] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:50.222575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106ff95-1c72-4861-a747-d88ebb559fc8 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:50.223270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106ff95-1c72-4861-a747-d88ebb559fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:50.223270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106ff95-1c72-4861-a747-d88ebb559fc8 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:50.224720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 697/1404] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:50.230114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0a634-95d1-4607-b025-e33c6e88a76d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:50.230591 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0a634-95d1-4607-b025-e33c6e88a76d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:50.230692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0a634-95d1-4607-b025-e33c6e88a76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:50.293024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:50.305840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cdb0a634-95d1-4607-b025-e33c6e88a76d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:50.319296 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cdb0a634-95d1-4607-b025-e33c6e88a76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:50.416866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 708/1405] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:50.705293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106ff95-1c72-4861-a747-d88ebb559fc8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:50.706800 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1106ff95-1c72-4861-a747-d88ebb559fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:50.851025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:50.887495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106ff95-1c72-4861-a747-d88ebb559fc8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:51.020713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fee7f-9194-4037-b1fe-51b5d38543d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:51.021360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fee7f-9194-4037-b1fe-51b5d38543d7 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:51.021701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fee7f-9194-4037-b1fe-51b5d38543d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:51.022040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fee7f-9194-4037-b1fe-51b5d38543d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:51.023740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-135fee7f-9194-4037-b1fe-51b5d38543d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:51.113144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:51.123905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fee7f-9194-4037-b1fe-51b5d38543d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:51.125642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fee7f-9194-4037-b1fe-51b5d38543d7 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:51.150689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 698/1406] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:24:53.439522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f972f4-ecd8-42c0-a069-b843f1a5371b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:53.440124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f972f4-ecd8-42c0-a069-b843f1a5371b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:53.440124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f972f4-ecd8-42c0-a069-b843f1a5371b None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:53.441829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 709/1407] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:24:53.447255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3021125-b22b-49da-9423-84519cc09a9c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:53.447682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3021125-b22b-49da-9423-84519cc09a9c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:53.447782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3021125-b22b-49da-9423-84519cc09a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:53.523437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b3021125-b22b-49da-9423-84519cc09a9c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:24:53.524833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b3021125-b22b-49da-9423-84519cc09a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:53.645984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 699/1408] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:24:53.948260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f972f4-ecd8-42c0-a069-b843f1a5371b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:24:53.949887 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-74f972f4-ecd8-42c0-a069-b843f1a5371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:54.098159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:54.128907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f972f4-ecd8-42c0-a069-b843f1a5371b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:24:54.263500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954c0f8-c937-446c-9680-55d7d8d106a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:24:54.264126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954c0f8-c937-446c-9680-55d7d8d106a0 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:24:54.264416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954c0f8-c937-446c-9680-55d7d8d106a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:24:54.264695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954c0f8-c937-446c-9680-55d7d8d106a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:24:54.267254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7954c0f8-c937-446c-9680-55d7d8d106a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:24:54.368760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:24:54.373552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954c0f8-c937-446c-9680-55d7d8d106a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:24:54.375215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954c0f8-c937-446c-9680-55d7d8d106a0 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:24:54.394871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 710/1409] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:03.642865 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=74 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:04.515438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=7 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:10.222601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1306ed-5f00-49f1-b25a-7b28a40e768b None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.223192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1306ed-5f00-49f1-b25a-7b28a40e768b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.223192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1306ed-5f00-49f1-b25a-7b28a40e768b None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.224511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 700/1410] 10.222.0.84 () {54 vars in 918 bytes} [Wed Aug 24 11:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:10.229629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2822b-a6e1-4d88-92ab-ab4abd4a46ac None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.236155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2822b-a6e1-4d88-92ab-ab4abd4a46ac None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.236155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2822b-a6e1-4d88-92ab-ab4abd4a46ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.307379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b7c2822b-a6e1-4d88-92ab-ab4abd4a46ac None None] MFA Rules not processed for user `c7cc35f5ddf0446e90ded5c85f21f37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:10.308572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b7c2822b-a6e1-4d88-92ab-ab4abd4a46ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:10.414103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 711/1411] 10.222.0.84 () {58 vars in 1015 bytes} [Wed Aug 24 11:25:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:10.428409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca96c96-c66e-426a-bcda-b8c668876525 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.428689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca96c96-c66e-426a-bcda-b8c668876525 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.428986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca96c96-c66e-426a-bcda-b8c668876525 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.430302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 701/1412] 10.222.0.84 () {54 vars in 918 bytes} [Wed Aug 24 11:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:10.435166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d8f1c-bcbd-4e26-a54f-a3b8af3252eb None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:10.435432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d8f1c-bcbd-4e26-a54f-a3b8af3252eb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:10.435697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d8f1c-bcbd-4e26-a54f-a3b8af3252eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:10.509732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:10.512636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6e3d8f1c-bcbd-4e26-a54f-a3b8af3252eb None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:10.513847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6e3d8f1c-bcbd-4e26-a54f-a3b8af3252eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:10.627288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 712/1413] 10.222.0.84 () {58 vars in 1015 bytes} [Wed Aug 24 11:25:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:13.026438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162221a-341a-46cb-991c-7d3d8b9a7525 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.026995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162221a-341a-46cb-991c-7d3d8b9a7525 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.026995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162221a-341a-46cb-991c-7d3d8b9a7525 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.028226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:13.033640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec73c13-f49f-4175-837c-060e8eda62f1 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.034090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec73c13-f49f-4175-837c-060e8eda62f1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.034203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec73c13-f49f-4175-837c-060e8eda62f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.106052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-8ec73c13-f49f-4175-837c-060e8eda62f1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:13.107262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8ec73c13-f49f-4175-837c-060e8eda62f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:13.163651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:13.205961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:13.499767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162221a-341a-46cb-991c-7d3d8b9a7525 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:13.501463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4162221a-341a-46cb-991c-7d3d8b9a7525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:13.645995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162221a-341a-46cb-991c-7d3d8b9a7525 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:13.719065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:13.759315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5ff95-b019-4c07-9efb-36d9f3e2a85e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:13.759847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5ff95-b019-4c07-9efb-36d9f3e2a85e service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:13.760122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5ff95-b019-4c07-9efb-36d9f3e2a85e service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:13.760387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5ff95-b019-4c07-9efb-36d9f3e2a85e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:13.762009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-36b5ff95-b019-4c07-9efb-36d9f3e2a85e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:13.853233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5ff95-b019-4c07-9efb-36d9f3e2a85e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:13.854825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5ff95-b019-4c07-9efb-36d9f3e2a85e service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:13.873943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 703/1416] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:14.771844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dc364-2c1b-4f20-8572-95a82462b863 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.772385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dc364-2c1b-4f20-8572-95a82462b863 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.772385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dc364-2c1b-4f20-8572-95a82462b863 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.773755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:14.778650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963f46d-dcb3-4775-8882-175484adb8f3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:14.779066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963f46d-dcb3-4775-8882-175484adb8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:14.779200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963f46d-dcb3-4775-8882-175484adb8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:14.856085 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d963f46d-dcb3-4775-8882-175484adb8f3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:14.857303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d963f46d-dcb3-4775-8882-175484adb8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:14.884498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:14.952396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:15.105650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dc364-2c1b-4f20-8572-95a82462b863 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:15.107184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cc8dc364-2c1b-4f20-8572-95a82462b863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:15.252555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dc364-2c1b-4f20-8572-95a82462b863 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:15.292364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:15.368579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0d5ba-771f-4a6c-b67b-75e6e96560a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:15.369212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0d5ba-771f-4a6c-b67b-75e6e96560a3 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:15.369490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0d5ba-771f-4a6c-b67b-75e6e96560a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:15.369757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0d5ba-771f-4a6c-b67b-75e6e96560a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:15.371365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-39f0d5ba-771f-4a6c-b67b-75e6e96560a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:15.461504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0d5ba-771f-4a6c-b67b-75e6e96560a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:15.463101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0d5ba-771f-4a6c-b67b-75e6e96560a3 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:15.481460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 715/1419] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:16.397498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040ffe-2890-44e4-9e44-8ba40ba48eb5 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:16.398068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040ffe-2890-44e4-9e44-8ba40ba48eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:16.398068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040ffe-2890-44e4-9e44-8ba40ba48eb5 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:16.399378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:16.404405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d5529-9d8e-45e1-9987-5d3e31ee8285 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:16.404845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d5529-9d8e-45e1-9987-5d3e31ee8285 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:16.404979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d5529-9d8e-45e1-9987-5d3e31ee8285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:16.459212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:16.483625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-1a9d5529-9d8e-45e1-9987-5d3e31ee8285 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:16.485520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1a9d5529-9d8e-45e1-9987-5d3e31ee8285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:16.584236 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:16.739503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040ffe-2890-44e4-9e44-8ba40ba48eb5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:16.741144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-68040ffe-2890-44e4-9e44-8ba40ba48eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:16.889852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040ffe-2890-44e4-9e44-8ba40ba48eb5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:16.902325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:17.004027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff01fd4-0820-4fb6-974f-118a58ee1940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:17.004536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff01fd4-0820-4fb6-974f-118a58ee1940 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:17.004833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff01fd4-0820-4fb6-974f-118a58ee1940 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:17.005105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff01fd4-0820-4fb6-974f-118a58ee1940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:17.006677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1ff01fd4-0820-4fb6-974f-118a58ee1940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:17.101255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff01fd4-0820-4fb6-974f-118a58ee1940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:17.102863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff01fd4-0820-4fb6-974f-118a58ee1940 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:17.121375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 706/1422] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:17.987386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6ee52-84a0-49ec-9995-3ad4b3374fe1 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:17.988213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6ee52-84a0-49ec-9995-3ad4b3374fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:17.988213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6ee52-84a0-49ec-9995-3ad4b3374fe1 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:17.989542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:17.994400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f5d61-7ec5-420d-a433-3bb99b830ad9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:17.994793 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f5d61-7ec5-420d-a433-3bb99b830ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:17.994936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f5d61-7ec5-420d-a433-3bb99b830ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:18.014548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:18.071003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-703f5d61-7ec5-420d-a433-3bb99b830ad9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:18.072210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-703f5d61-7ec5-420d-a433-3bb99b830ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:18.168551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:18.318223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6ee52-84a0-49ec-9995-3ad4b3374fe1 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:18.320428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a5e6ee52-84a0-49ec-9995-3ad4b3374fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:18.432637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:18.469059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6ee52-84a0-49ec-9995-3ad4b3374fe1 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:18.581589 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58f73a-24a1-4aba-a2d9-f77a69defc37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:18.582196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58f73a-24a1-4aba-a2d9-f77a69defc37 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:18.582470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58f73a-24a1-4aba-a2d9-f77a69defc37 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:18.582737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58f73a-24a1-4aba-a2d9-f77a69defc37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:18.584328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2f58f73a-24a1-4aba-a2d9-f77a69defc37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:18.660649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:18.673967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58f73a-24a1-4aba-a2d9-f77a69defc37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:18.675560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58f73a-24a1-4aba-a2d9-f77a69defc37 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:18.693671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 718/1425] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:19.557363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2f2ef-b07e-4151-a7dc-aeeff937770d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.559476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2f2ef-b07e-4151-a7dc-aeeff937770d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.559476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2f2ef-b07e-4151-a7dc-aeeff937770d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.559476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:19.564198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac480c62-3544-4961-8cdf-ae9b09b8d127 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:19.564647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac480c62-3544-4961-8cdf-ae9b09b8d127 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:19.564766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac480c62-3544-4961-8cdf-ae9b09b8d127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:19.640840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ac480c62-3544-4961-8cdf-ae9b09b8d127 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:19.642053 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ac480c62-3544-4961-8cdf-ae9b09b8d127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:19.740570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:19.894824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2f2ef-b07e-4151-a7dc-aeeff937770d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:19.896367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-04e2f2ef-b07e-4151-a7dc-aeeff937770d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:19.979787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:20.042486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2f2ef-b07e-4151-a7dc-aeeff937770d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:20.156009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493e72a-9548-49d3-b3bc-e01e2bf901cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:20.156561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493e72a-9548-49d3-b3bc-e01e2bf901cf service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:20.156864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493e72a-9548-49d3-b3bc-e01e2bf901cf service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:20.157134 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493e72a-9548-49d3-b3bc-e01e2bf901cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:20.158760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0493e72a-9548-49d3-b3bc-e01e2bf901cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:20.213934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:20.254751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493e72a-9548-49d3-b3bc-e01e2bf901cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:20.256440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493e72a-9548-49d3-b3bc-e01e2bf901cf service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:20.275182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 709/1428] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:21.213497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20359a2e-1410-440f-91e7-2ab0b3749fbb None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.214046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20359a2e-1410-440f-91e7-2ab0b3749fbb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.214046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20359a2e-1410-440f-91e7-2ab0b3749fbb None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.215385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:21.221008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbec3d-6d61-4f11-90ec-3fb3ebd882f8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.221451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbec3d-6d61-4f11-90ec-3fb3ebd882f8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.221556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbec3d-6d61-4f11-90ec-3fb3ebd882f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.297813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9fbbec3d-6d61-4f11-90ec-3fb3ebd882f8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:21.299026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9fbbec3d-6d61-4f11-90ec-3fb3ebd882f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:21.395135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:21.550300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20359a2e-1410-440f-91e7-2ab0b3749fbb None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:21.551756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-20359a2e-1410-440f-91e7-2ab0b3749fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:21.600186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:21.696705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20359a2e-1410-440f-91e7-2ab0b3749fbb None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:21.811183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcab7a2-d87d-446d-bec1-c47db77fd4c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:21.811780 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcab7a2-d87d-446d-bec1-c47db77fd4c0 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:21.812726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcab7a2-d87d-446d-bec1-c47db77fd4c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:21.813017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcab7a2-d87d-446d-bec1-c47db77fd4c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:21.814636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0bcab7a2-d87d-446d-bec1-c47db77fd4c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:21.821570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:21.902399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcab7a2-d87d-446d-bec1-c47db77fd4c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:21.904089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcab7a2-d87d-446d-bec1-c47db77fd4c0 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:21.923104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 721/1431] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:22.793962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596958c9-b4b0-40df-ae7c-b5027e1be2eb None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:22.794567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596958c9-b4b0-40df-ae7c-b5027e1be2eb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:22.794866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596958c9-b4b0-40df-ae7c-b5027e1be2eb None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:22.797376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:25:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:22.802876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a108a-a343-48f2-9244-17b9524b7439 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:22.803426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a108a-a343-48f2-9244-17b9524b7439 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:22.803426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a108a-a343-48f2-9244-17b9524b7439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:22.885367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ac8a108a-a343-48f2-9244-17b9524b7439 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:22.886564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ac8a108a-a343-48f2-9244-17b9524b7439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:22.955586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:22.983572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:23.136708 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596958c9-b4b0-40df-ae7c-b5027e1be2eb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:23.138244 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-596958c9-b4b0-40df-ae7c-b5027e1be2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:23.160493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:23.302291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596958c9-b4b0-40df-ae7c-b5027e1be2eb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:23.389132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:23.414765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa582100-22c8-4711-9b72-ebf282f03197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:23.415359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa582100-22c8-4711-9b72-ebf282f03197 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:23.415632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa582100-22c8-4711-9b72-ebf282f03197 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:23.415893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa582100-22c8-4711-9b72-ebf282f03197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:23.417520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aa582100-22c8-4711-9b72-ebf282f03197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:23.506414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa582100-22c8-4711-9b72-ebf282f03197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:23.509406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa582100-22c8-4711-9b72-ebf282f03197 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:23.531456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 712/1434] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:24.443993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3aaa6-7519-45ad-9884-a20eff0222b6 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.444565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3aaa6-7519-45ad-9884-a20eff0222b6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.444565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3aaa6-7519-45ad-9884-a20eff0222b6 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.446049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:24.451118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fab90b-a1c9-4f26-8cef-cbe1d5797c59 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:24.451537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fab90b-a1c9-4f26-8cef-cbe1d5797c59 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:24.451695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fab90b-a1c9-4f26-8cef-cbe1d5797c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:24.528897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a8fab90b-a1c9-4f26-8cef-cbe1d5797c59 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:24.530109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a8fab90b-a1c9-4f26-8cef-cbe1d5797c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:24.579025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:24.635136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:24.787060 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3aaa6-7519-45ad-9884-a20eff0222b6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:24.788560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-47a3aaa6-7519-45ad-9884-a20eff0222b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:24.942664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3aaa6-7519-45ad-9884-a20eff0222b6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:24.997977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:25.055234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a1265-ef2f-46b2-a7a0-cad66e5a861c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:25.055832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a1265-ef2f-46b2-a7a0-cad66e5a861c service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:25.056101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a1265-ef2f-46b2-a7a0-cad66e5a861c service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:25.056361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a1265-ef2f-46b2-a7a0-cad66e5a861c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:25.057976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c34a1265-ef2f-46b2-a7a0-cad66e5a861c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:25.146582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a1265-ef2f-46b2-a7a0-cad66e5a861c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:25.148194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a1265-ef2f-46b2-a7a0-cad66e5a861c service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:25.166507 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 724/1437] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:26.054318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e8779-a4c8-46f4-9ed7-e991acb46293 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.054833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e8779-a4c8-46f4-9ed7-e991acb46293 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.054833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e8779-a4c8-46f4-9ed7-e991acb46293 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.056184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:26.061105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60a63a-7ef5-4dcc-a1f6-f04d6a8c316c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.061559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60a63a-7ef5-4dcc-a1f6-f04d6a8c316c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.061665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60a63a-7ef5-4dcc-a1f6-f04d6a8c316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.137478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ce60a63a-7ef5-4dcc-a1f6-f04d6a8c316c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:26.138719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ce60a63a-7ef5-4dcc-a1f6-f04d6a8c316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:26.141878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:26.236601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:26.386475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e8779-a4c8-46f4-9ed7-e991acb46293 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:26.388061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bb2e8779-a4c8-46f4-9ed7-e991acb46293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:26.537718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e8779-a4c8-46f4-9ed7-e991acb46293 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:26.568100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:26.658776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bc1a7-eedd-4fe9-b95d-f92498835d22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:26.659390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bc1a7-eedd-4fe9-b95d-f92498835d22 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:26.659662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bc1a7-eedd-4fe9-b95d-f92498835d22 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:26.659927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bc1a7-eedd-4fe9-b95d-f92498835d22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:26.661560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8d3bc1a7-eedd-4fe9-b95d-f92498835d22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:26.752789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bc1a7-eedd-4fe9-b95d-f92498835d22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:26.754391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bc1a7-eedd-4fe9-b95d-f92498835d22 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:26.772001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 715/1440] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:28.632589 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecbea7-2f14-4e5b-a705-aea3fc828504 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:28.633200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecbea7-2f14-4e5b-a705-aea3fc828504 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:28.633200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecbea7-2f14-4e5b-a705-aea3fc828504 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:28.634511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:28.640846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d4e54-7644-4a08-be87-e8efcd37c1ed None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:28.641308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d4e54-7644-4a08-be87-e8efcd37c1ed None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:28.641418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d4e54-7644-4a08-be87-e8efcd37c1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:28.684226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:28.709815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b83d4e54-7644-4a08-be87-e8efcd37c1ed None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:28.711297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b83d4e54-7644-4a08-be87-e8efcd37c1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:28.810909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:29.106289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecbea7-2f14-4e5b-a705-aea3fc828504 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:29.107962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-54ecbea7-2f14-4e5b-a705-aea3fc828504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:29.216379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecbea7-2f14-4e5b-a705-aea3fc828504 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:29.258415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:29.300914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12425608-a361-4640-8413-4a2b130f1390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:29.301536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12425608-a361-4640-8413-4a2b130f1390 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:29.301818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12425608-a361-4640-8413-4a2b130f1390 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:29.302087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12425608-a361-4640-8413-4a2b130f1390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:29.303718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-12425608-a361-4640-8413-4a2b130f1390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:29.410100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12425608-a361-4640-8413-4a2b130f1390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:29.411526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12425608-a361-4640-8413-4a2b130f1390 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:29.430117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 727/1443] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:31.550709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e46f44-9aab-4d2b-9f0d-9f99436491ec None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:31.551280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e46f44-9aab-4d2b-9f0d-9f99436491ec None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:31.551280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e46f44-9aab-4d2b-9f0d-9f99436491ec None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:31.552595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:31.558112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c130566-fb1c-4682-82d4-ecb6d5450726 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:31.558543 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c130566-fb1c-4682-82d4-ecb6d5450726 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:31.558683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c130566-fb1c-4682-82d4-ecb6d5450726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:31.624820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-5c130566-fb1c-4682-82d4-ecb6d5450726 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:31.626020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5c130566-fb1c-4682-82d4-ecb6d5450726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:31.638594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:31.724330 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:32.012429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e46f44-9aab-4d2b-9f0d-9f99436491ec None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:32.014023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a7e46f44-9aab-4d2b-9f0d-9f99436491ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:32.120445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e46f44-9aab-4d2b-9f0d-9f99436491ec None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:32.133494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:32.195226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfed7e-0074-44b6-9a1f-7347b81e1bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:32.195837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfed7e-0074-44b6-9a1f-7347b81e1bcf service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:32.196115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfed7e-0074-44b6-9a1f-7347b81e1bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:32.196388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfed7e-0074-44b6-9a1f-7347b81e1bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:32.198059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2dbfed7e-0074-44b6-9a1f-7347b81e1bcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:32.308776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfed7e-0074-44b6-9a1f-7347b81e1bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:32.310228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfed7e-0074-44b6-9a1f-7347b81e1bcf service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:32.328540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 718/1446] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:34.615052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d10a89-46ee-4452-8d97-034f46b86a1d None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:34.615647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d10a89-46ee-4452-8d97-034f46b86a1d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:34.615647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d10a89-46ee-4452-8d97-034f46b86a1d None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:34.617006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:34.622508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63649fc0-55e4-4ded-991e-88f989fd6eed None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:34.622945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63649fc0-55e4-4ded-991e-88f989fd6eed None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:34.623148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63649fc0-55e4-4ded-991e-88f989fd6eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:34.674231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:34.693313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-63649fc0-55e4-4ded-991e-88f989fd6eed None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:34.694586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-63649fc0-55e4-4ded-991e-88f989fd6eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:34.805736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:35.098109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d10a89-46ee-4452-8d97-034f46b86a1d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:35.099645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d9d10a89-46ee-4452-8d97-034f46b86a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:35.203583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d10a89-46ee-4452-8d97-034f46b86a1d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:35.243309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:35.280844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc3d7c-2712-4aba-bf9e-38ba62789116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:35.281449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc3d7c-2712-4aba-bf9e-38ba62789116 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:35.281727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc3d7c-2712-4aba-bf9e-38ba62789116 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:35.281996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc3d7c-2712-4aba-bf9e-38ba62789116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:35.283610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-bccc3d7c-2712-4aba-bf9e-38ba62789116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:35.395402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc3d7c-2712-4aba-bf9e-38ba62789116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:35.397192 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc3d7c-2712-4aba-bf9e-38ba62789116 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:35.415806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 730/1449] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:37.468645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31daf6-84a7-4596-add0-361db3f9c73f None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:37.469261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31daf6-84a7-4596-add0-361db3f9c73f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:37.469261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31daf6-84a7-4596-add0-361db3f9c73f None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:37.470566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:37.475877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaf2b7-2d12-4418-83de-dfe6ef3af541 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:37.476274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaf2b7-2d12-4418-83de-dfe6ef3af541 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:37.476423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaf2b7-2d12-4418-83de-dfe6ef3af541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:37.542721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-eddaf2b7-2d12-4418-83de-dfe6ef3af541 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:37.544010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-eddaf2b7-2d12-4418-83de-dfe6ef3af541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:37.565925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:37.641274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:37.927365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31daf6-84a7-4596-add0-361db3f9c73f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:37.928957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ff31daf6-84a7-4596-add0-361db3f9c73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:38.037280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31daf6-84a7-4596-add0-361db3f9c73f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:38.057993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:38.114209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64c31e-66cd-4bea-be80-4d792077d1eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:38.114848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64c31e-66cd-4bea-be80-4d792077d1eb service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:38.115121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64c31e-66cd-4bea-be80-4d792077d1eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:38.115390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64c31e-66cd-4bea-be80-4d792077d1eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:38.117019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3b64c31e-66cd-4bea-be80-4d792077d1eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:38.230295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64c31e-66cd-4bea-be80-4d792077d1eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:38.231901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64c31e-66cd-4bea-be80-4d792077d1eb service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:38.250962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 721/1452] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:40.330467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614ab05-bc0b-4876-9879-091227eb55bd None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:40.331090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614ab05-bc0b-4876-9879-091227eb55bd None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:40.331090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614ab05-bc0b-4876-9879-091227eb55bd None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:40.332342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:25:40.337595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9efed-d1c0-4ecd-b270-05074a0cb784 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:40.338014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9efed-d1c0-4ecd-b270-05074a0cb784 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:40.338167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9efed-d1c0-4ecd-b270-05074a0cb784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:40.405357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:40.408038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-14b9efed-d1c0-4ecd-b270-05074a0cb784 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:40.409294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-14b9efed-d1c0-4ecd-b270-05074a0cb784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:40.506534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:40.792432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614ab05-bc0b-4876-9879-091227eb55bd None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:40.793998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7614ab05-bc0b-4876-9879-091227eb55bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:40.896333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614ab05-bc0b-4876-9879-091227eb55bd None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:40.945372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:40.969623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d9e62-85c1-4496-9d25-9f733007c24c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:40.970223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d9e62-85c1-4496-9d25-9f733007c24c service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:40.970497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d9e62-85c1-4496-9d25-9f733007c24c service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:40.970766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d9e62-85c1-4496-9d25-9f733007c24c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:40.972344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-094d9e62-85c1-4496-9d25-9f733007c24c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:41.078278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d9e62-85c1-4496-9d25-9f733007c24c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:41.079699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d9e62-85c1-4496-9d25-9f733007c24c service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:41.099299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 733/1455] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:43.067076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50b50-32ea-45b0-97f8-461e9b1f4ef6 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:43.067628 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50b50-32ea-45b0-97f8-461e9b1f4ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:43.067628 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50b50-32ea-45b0-97f8-461e9b1f4ef6 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:43.069004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:43.074520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d983b8-f303-4441-95b9-79e777beeeea None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:43.074949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d983b8-f303-4441-95b9-79e777beeeea None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:43.075051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d983b8-f303-4441-95b9-79e777beeeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:43.141201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-76d983b8-f303-4441-95b9-79e777beeeea None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:43.142401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-76d983b8-f303-4441-95b9-79e777beeeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:43.171364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:43.237113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:25:43.522437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b50b50-32ea-45b0-97f8-461e9b1f4ef6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:43.523949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a1b50b50-32ea-45b0-97f8-461e9b1f4ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:43.628312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b50b50-32ea-45b0-97f8-461e9b1f4ef6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:43.656356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:43.703093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d1f99-21ff-45ae-bd2a-40eb7529d8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:43.703689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d1f99-21ff-45ae-bd2a-40eb7529d8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:43.703961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d1f99-21ff-45ae-bd2a-40eb7529d8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:43.704227 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d1f99-21ff-45ae-bd2a-40eb7529d8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:43.706012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-562d1f99-21ff-45ae-bd2a-40eb7529d8d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:43.815131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d1f99-21ff-45ae-bd2a-40eb7529d8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:43.816570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d1f99-21ff-45ae-bd2a-40eb7529d8d0 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:43.834443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 724/1458] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:45.821790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d7ffd-6380-40a8-9195-de98fe48eb8a None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:45.822333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d7ffd-6380-40a8-9195-de98fe48eb8a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:45.822444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d7ffd-6380-40a8-9195-de98fe48eb8a None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:45.823749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:45.829478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eac46-56c7-49b7-9593-a280d4a08961 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:45.829947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eac46-56c7-49b7-9593-a280d4a08961 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:45.830105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236eac46-56c7-49b7-9593-a280d4a08961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:45.896363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-236eac46-56c7-49b7-9593-a280d4a08961 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:45.897606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-236eac46-56c7-49b7-9593-a280d4a08961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:45.906448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:45.997499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:46.278412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d7ffd-6380-40a8-9195-de98fe48eb8a None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:46.285571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-660d7ffd-6380-40a8-9195-de98fe48eb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:46.381677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d7ffd-6380-40a8-9195-de98fe48eb8a None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:46.435214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:46.454167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ec7d8-1d84-4d01-baa8-911a1aa6958b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:46.454760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ec7d8-1d84-4d01-baa8-911a1aa6958b service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:46.455031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ec7d8-1d84-4d01-baa8-911a1aa6958b service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:46.455291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ec7d8-1d84-4d01-baa8-911a1aa6958b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:46.456885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-655ec7d8-1d84-4d01-baa8-911a1aa6958b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:46.563050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ec7d8-1d84-4d01-baa8-911a1aa6958b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:46.564468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ec7d8-1d84-4d01-baa8-911a1aa6958b service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:46.582684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 736/1461] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:48.545763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c7747-3959-4b06-a739-f6b0ff5aa1a3 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:48.546276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c7747-3959-4b06-a739-f6b0ff5aa1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:48.546276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c7747-3959-4b06-a739-f6b0ff5aa1a3 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:48.547550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:48.552921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d66fae-5eaa-43ac-8eb7-f238af433292 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:48.553388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d66fae-5eaa-43ac-8eb7-f238af433292 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:48.553494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d66fae-5eaa-43ac-8eb7-f238af433292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:48.619832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c4d66fae-5eaa-43ac-8eb7-f238af433292 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:48.621057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c4d66fae-5eaa-43ac-8eb7-f238af433292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:48.661061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:48.718822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:49.000173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c7747-3959-4b06-a739-f6b0ff5aa1a3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:49.001745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b49c7747-3959-4b06-a739-f6b0ff5aa1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:49.104984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c7747-3959-4b06-a739-f6b0ff5aa1a3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:49.149187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:49.185892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb340b-e45a-4c04-a47d-c75947d5b00f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:49.186489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb340b-e45a-4c04-a47d-c75947d5b00f service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:49.186761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb340b-e45a-4c04-a47d-c75947d5b00f service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:49.187025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb340b-e45a-4c04-a47d-c75947d5b00f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:49.188598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-defb340b-e45a-4c04-a47d-c75947d5b00f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:49.296145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb340b-e45a-4c04-a47d-c75947d5b00f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:49.297614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb340b-e45a-4c04-a47d-c75947d5b00f service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:49.315636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 727/1464] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:25:51.233679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213060f-7dc0-4bf8-bdcf-beefdd7e6f80 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:51.234216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213060f-7dc0-4bf8-bdcf-beefdd7e6f80 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:51.234216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213060f-7dc0-4bf8-bdcf-beefdd7e6f80 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:51.235605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:51.241381 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07249-14fc-4c83-8b07-fad61e6c21d3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:51.241788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07249-14fc-4c83-8b07-fad61e6c21d3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:51.241929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba07249-14fc-4c83-8b07-fad61e6c21d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:51.308688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1ba07249-14fc-4c83-8b07-fad61e6c21d3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:51.309906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1ba07249-14fc-4c83-8b07-fad61e6c21d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:51.329902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:51.408386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:51.702583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213060f-7dc0-4bf8-bdcf-beefdd7e6f80 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:51.704121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a213060f-7dc0-4bf8-bdcf-beefdd7e6f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:51.808467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213060f-7dc0-4bf8-bdcf-beefdd7e6f80 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:51.875166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:51.881673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05f07f-5d05-4f38-8b1b-9971b3a657c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:51.882272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05f07f-5d05-4f38-8b1b-9971b3a657c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:51.882542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05f07f-5d05-4f38-8b1b-9971b3a657c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:51.882811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05f07f-5d05-4f38-8b1b-9971b3a657c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:51.884411 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7a05f07f-5d05-4f38-8b1b-9971b3a657c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:51.991347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05f07f-5d05-4f38-8b1b-9971b3a657c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:51.992688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05f07f-5d05-4f38-8b1b-9971b3a657c6 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:52.011581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 739/1467] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:25:53.954359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae54141-4535-4190-ba99-6936863a6bf5 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:53.954880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae54141-4535-4190-ba99-6936863a6bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:53.954880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae54141-4535-4190-ba99-6936863a6bf5 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:53.956285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:25:53.961751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60429f3-14f6-49a8-b76e-90a134cf1825 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:53.962196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60429f3-14f6-49a8-b76e-90a134cf1825 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:53.962292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60429f3-14f6-49a8-b76e-90a134cf1825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:54.029312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a60429f3-14f6-49a8-b76e-90a134cf1825 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:25:54.030547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a60429f3-14f6-49a8-b76e-90a134cf1825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:54.079894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:54.127978 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:25:54.410598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae54141-4535-4190-ba99-6936863a6bf5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:25:54.412136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4ae54141-4535-4190-ba99-6936863a6bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:54.661414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae54141-4535-4190-ba99-6936863a6bf5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:25:54.708459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:25:54.734774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc066642-2c52-46ce-a1fa-b9f67bf749ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:25:54.735370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc066642-2c52-46ce-a1fa-b9f67bf749ef service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:25:54.735644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc066642-2c52-46ce-a1fa-b9f67bf749ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:25:54.735911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc066642-2c52-46ce-a1fa-b9f67bf749ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:25:54.737578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cc066642-2c52-46ce-a1fa-b9f67bf749ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:25:54.845361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc066642-2c52-46ce-a1fa-b9f67bf749ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:25:54.846787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc066642-2c52-46ce-a1fa-b9f67bf749ef service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:25:54.865965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 730/1470] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:04.129731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=22 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:04.864018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=66 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:14.769617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5757b0-577a-444a-ae52-eac03cc83cc0 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:14.770131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5757b0-577a-444a-ae52-eac03cc83cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:14.770131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5757b0-577a-444a-ae52-eac03cc83cc0 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:14.771469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 741/1471] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:26:14.776664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a36ee-904f-44cc-a75a-8391f063a5ac None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:14.777094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a36ee-904f-44cc-a75a-8391f063a5ac None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:14.777236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a36ee-904f-44cc-a75a-8391f063a5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:14.841769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-422a36ee-904f-44cc-a75a-8391f063a5ac None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:14.842870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-422a36ee-904f-44cc-a75a-8391f063a5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:14.939294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 731/1472] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:15.239656 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fcda4-1746-433c-aa49-0d901dcd24ce None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:15.239937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fcda4-1746-433c-aa49-0d901dcd24ce None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:15.240209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fcda4-1746-433c-aa49-0d901dcd24ce None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:15.241626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 742/1473] 10.222.0.84 () {54 vars in 967 bytes} [Wed Aug 24 11:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:26:15.246386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6db8cf-b095-43cb-a358-ba5695956256 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:15.246671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6db8cf-b095-43cb-a358-ba5695956256 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:15.246948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6db8cf-b095-43cb-a358-ba5695956256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:15.327025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4d6db8cf-b095-43cb-a358-ba5695956256 None None] MFA Rules not processed for user `75ac781cf2d84d95a789724e74e12acf`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:15.328243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4d6db8cf-b095-43cb-a358-ba5695956256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:15.349588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:15.406599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 732/1474] 10.222.0.84 () {58 vars in 1064 bytes} [Wed Aug 24 11:26:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:15.414217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fcda4-1746-433c-aa49-0d901dcd24ce None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:15.415776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e33fcda4-1746-433c-aa49-0d901dcd24ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:15.521407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fcda4-1746-433c-aa49-0d901dcd24ce None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:15.592925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f9af4-ba86-4d9f-ae20-8744dcddd53d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ac781cf2d84d95a789724e74e12acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:15.593526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f9af4-ba86-4d9f-ae20-8744dcddd53d service glance] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:15.593801 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f9af4-ba86-4d9f-ae20-8744dcddd53d service glance] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:15.594065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f9af4-ba86-4d9f-ae20-8744dcddd53d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:15.595641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-328f9af4-ba86-4d9f-ae20-8744dcddd53d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:15.651044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:15.708498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f9af4-ba86-4d9f-ae20-8744dcddd53d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:15.710657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f9af4-ba86-4d9f-ae20-8744dcddd53d service glance] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:15.729555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 743/1475] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:17.765483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf7646-1769-4194-bc7a-caac02e41c00 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.766012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf7646-1769-4194-bc7a-caac02e41c00 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.766012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf7646-1769-4194-bc7a-caac02e41c00 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.767270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 733/1476] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:26:17.772691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57000bbe-ebf1-4929-b5a0-480754b8e36d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:17.773131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57000bbe-ebf1-4929-b5a0-480754b8e36d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:17.773234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57000bbe-ebf1-4929-b5a0-480754b8e36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:17.840158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-57000bbe-ebf1-4929-b5a0-480754b8e36d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:17.841403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-57000bbe-ebf1-4929-b5a0-480754b8e36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:17.939528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 744/1477] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:18.219730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf7646-1769-4194-bc7a-caac02e41c00 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:18.221421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8ddf7646-1769-4194-bc7a-caac02e41c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:18.348903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf7646-1769-4194-bc7a-caac02e41c00 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:18.397998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:18.442788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df921626-13b4-4a9b-bb06-b818c9639099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:18.443415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df921626-13b4-4a9b-bb06-b818c9639099 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:18.443689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df921626-13b4-4a9b-bb06-b818c9639099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:18.443952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df921626-13b4-4a9b-bb06-b818c9639099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:18.444582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df921626-13b4-4a9b-bb06-b818c9639099 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:18.446334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-df921626-13b4-4a9b-bb06-b818c9639099 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:26:18.446334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Aug 24 11:26:18.446334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: Aug 24 11:26:18.447472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df921626-13b4-4a9b-bb06-b818c9639099 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:18.467058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 734/1478] 10.222.0.84 () {56 vars in 1108 bytes} [Wed Aug 24 11:26:18 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:26:20.418843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b44a9-8f41-45d8-9e57-037563b7a705 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:20.419415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b44a9-8f41-45d8-9e57-037563b7a705 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:20.419415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b44a9-8f41-45d8-9e57-037563b7a705 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:20.420945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 745/1479] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:26:20.426170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb7ee8-50e4-49fe-bebf-a5ff800a63a7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:20.426554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb7ee8-50e4-49fe-bebf-a5ff800a63a7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:20.426718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb7ee8-50e4-49fe-bebf-a5ff800a63a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:20.493152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e3fb7ee8-50e4-49fe-bebf-a5ff800a63a7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:20.494343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e3fb7ee8-50e4-49fe-bebf-a5ff800a63a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:20.593299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 735/1480] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:20.888612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b44a9-8f41-45d8-9e57-037563b7a705 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:20.890177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a35b44a9-8f41-45d8-9e57-037563b7a705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:20.899866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:21.035043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b44a9-8f41-45d8-9e57-037563b7a705 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:21.114537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:21.147700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda94c01-04e6-4f77-ab6d-15b7cb44be39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:21.148306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94c01-04e6-4f77-ab6d-15b7cb44be39 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:21.148585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94c01-04e6-4f77-ab6d-15b7cb44be39 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:21.148877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda94c01-04e6-4f77-ab6d-15b7cb44be39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:21.150468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fda94c01-04e6-4f77-ab6d-15b7cb44be39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:21.240647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda94c01-04e6-4f77-ab6d-15b7cb44be39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:21.242378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda94c01-04e6-4f77-ab6d-15b7cb44be39 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:21.260390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 746/1481] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:23.406891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b947f-add6-4610-9a5d-2d8b98e3571c None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:23.407475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b947f-add6-4610-9a5d-2d8b98e3571c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:23.407475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b947f-add6-4610-9a5d-2d8b98e3571c None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:23.408740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 736/1482] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:26:23.413939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf4fc3-072c-4756-89b1-ae17f8f90211 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:23.414359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf4fc3-072c-4756-89b1-ae17f8f90211 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:23.414462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf4fc3-072c-4756-89b1-ae17f8f90211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:23.479635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ccdf4fc3-072c-4756-89b1-ae17f8f90211 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:23.480848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ccdf4fc3-072c-4756-89b1-ae17f8f90211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:23.527376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:23.578156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 747/1483] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:23.872350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b947f-add6-4610-9a5d-2d8b98e3571c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:23.873905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f65b947f-add6-4610-9a5d-2d8b98e3571c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:23.897132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:23.979782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b947f-add6-4610-9a5d-2d8b98e3571c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:24.063601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa96366-2fe9-495d-a35c-9dffc02ed838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:24.064202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96366-2fe9-495d-a35c-9dffc02ed838 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:24.064472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96366-2fe9-495d-a35c-9dffc02ed838 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:24.064849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa96366-2fe9-495d-a35c-9dffc02ed838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:24.066437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9aa96366-2fe9-495d-a35c-9dffc02ed838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:24.141409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:24.176408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa96366-2fe9-495d-a35c-9dffc02ed838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:24.178333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa96366-2fe9-495d-a35c-9dffc02ed838 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:24.197941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 737/1484] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:26.210346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44693272-bd68-42fb-8cb2-2e0dff1a9394 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:26.210912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44693272-bd68-42fb-8cb2-2e0dff1a9394 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:26.210912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44693272-bd68-42fb-8cb2-2e0dff1a9394 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:26.212283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 748/1485] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:26:26.218132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31d350-4a85-47ff-9dc3-6d347add9164 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:26.218544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31d350-4a85-47ff-9dc3-6d347add9164 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:26.218886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31d350-4a85-47ff-9dc3-6d347add9164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:26.284303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6e31d350-4a85-47ff-9dc3-6d347add9164 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:26.285538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6e31d350-4a85-47ff-9dc3-6d347add9164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:26.382157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 738/1486] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:26.675380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44693272-bd68-42fb-8cb2-2e0dff1a9394 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:26.676934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-44693272-bd68-42fb-8cb2-2e0dff1a9394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:26.783288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44693272-bd68-42fb-8cb2-2e0dff1a9394 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:26.854368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:26.856110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eac5b9-075f-4487-a358-75eac3c1164f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:26.856704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eac5b9-075f-4487-a358-75eac3c1164f service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:26.857148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eac5b9-075f-4487-a358-75eac3c1164f service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:26.857453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eac5b9-075f-4487-a358-75eac3c1164f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:26.859034 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-88eac5b9-075f-4487-a358-75eac3c1164f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:26.965939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eac5b9-075f-4487-a358-75eac3c1164f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:26.967428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eac5b9-075f-4487-a358-75eac3c1164f service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:26.985453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 749/1487] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:28.991293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a671f-8742-44f2-a741-70367ad9a36a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:28.991920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a671f-8742-44f2-a741-70367ad9a36a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:28.991920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a671f-8742-44f2-a741-70367ad9a36a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:28.993141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 739/1488] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:26:28.998733 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf1afa-e99e-4d6c-bb69-652ddfd81887 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:28.999146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf1afa-e99e-4d6c-bb69-652ddfd81887 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:28.999294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf1afa-e99e-4d6c-bb69-652ddfd81887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:29.065278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cbcf1afa-e99e-4d6c-bb69-652ddfd81887 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:29.066467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cbcf1afa-e99e-4d6c-bb69-652ddfd81887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:29.122626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:29.166467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 750/1489] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:29.458017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a671f-8742-44f2-a741-70367ad9a36a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:29.459529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-086a671f-8742-44f2-a741-70367ad9a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:29.495374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:29.564615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a671f-8742-44f2-a741-70367ad9a36a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:29.640969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be226616-c3e1-441e-bda9-f1f4f4c011ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:29.641564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be226616-c3e1-441e-bda9-f1f4f4c011ad service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:29.641835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be226616-c3e1-441e-bda9-f1f4f4c011ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:29.642108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be226616-c3e1-441e-bda9-f1f4f4c011ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:29.643698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-be226616-c3e1-441e-bda9-f1f4f4c011ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:29.724710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:29.753164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be226616-c3e1-441e-bda9-f1f4f4c011ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:29.754938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be226616-c3e1-441e-bda9-f1f4f4c011ad service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:29.773459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 740/1490] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:32.098186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773cb07-9fef-4d93-a366-1299a8fa02ff None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:32.098746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773cb07-9fef-4d93-a366-1299a8fa02ff None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:32.098746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773cb07-9fef-4d93-a366-1299a8fa02ff None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:32.100057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 751/1491] 10.222.0.84 () {54 vars in 927 bytes} [Wed Aug 24 11:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:26:32.105237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f41d2-adc9-40fa-a908-4a27b844344b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:32.105638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f41d2-adc9-40fa-a908-4a27b844344b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:32.105790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f41d2-adc9-40fa-a908-4a27b844344b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:32.173721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-755f41d2-adc9-40fa-a908-4a27b844344b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:26:32.174929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-755f41d2-adc9-40fa-a908-4a27b844344b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:32.273978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 741/1492] 10.222.0.84 () {58 vars in 1024 bytes} [Wed Aug 24 11:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:26:32.570963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773cb07-9fef-4d93-a366-1299a8fa02ff None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:26:32.572472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b773cb07-9fef-4d93-a366-1299a8fa02ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:32.680523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773cb07-9fef-4d93-a366-1299a8fa02ff None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:26:32.754615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353feab-04a8-414d-a797-351ae38916af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:26:32.755223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353feab-04a8-414d-a797-351ae38916af service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:26:32.755501 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353feab-04a8-414d-a797-351ae38916af service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:26:32.755769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353feab-04a8-414d-a797-351ae38916af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:26:32.757414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4353feab-04a8-414d-a797-351ae38916af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:26:32.767144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:32.864826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353feab-04a8-414d-a797-351ae38916af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:26:32.866189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353feab-04a8-414d-a797-351ae38916af service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:26:32.884654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 752/1493] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:26:42.272062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=82 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:26:42.882358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=50 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:54.770617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41315d10-fb00-49d0-ad2f-e26d7c6465af None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:54.771214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41315d10-fb00-49d0-ad2f-e26d7c6465af None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:54.771214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41315d10-fb00-49d0-ad2f-e26d7c6465af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:54.846541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-41315d10-fb00-49d0-ad2f-e26d7c6465af None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:54.847789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-41315d10-fb00-49d0-ad2f-e26d7c6465af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:54.945986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 742/1494] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:54.954732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade45c0-cdb6-4323-be23-1f31108ef068 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:54.955221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade45c0-cdb6-4323-be23-1f31108ef068 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:54.955323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade45c0-cdb6-4323-be23-1f31108ef068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:55.031176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-5ade45c0-cdb6-4323-be23-1f31108ef068 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:55.032395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5ade45c0-cdb6-4323-be23-1f31108ef068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:55.133049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 753/1495] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:55.137802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41315d10-fb00-49d0-ad2f-e26d7c6465af None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:55.139732 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-41315d10-fb00-49d0-ad2f-e26d7c6465af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:55.150970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41315d10-fb00-49d0-ad2f-e26d7c6465af None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:55.211856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:55.245899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:55.246484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:55.246754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:55.247018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:55.247643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:27:55.248597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:55.249509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:27:55.249509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Aug 24 11:27:55.249509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: Aug 24 11:27:55.250457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:55.261974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 743/1496] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:27:55.270436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade45c0-cdb6-4323-be23-1f31108ef068 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:55.279836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade45c0-cdb6-4323-be23-1f31108ef068 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:55.340849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:55.375202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:55.375802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:55.376074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:55.376345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:55.376989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:55.378752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WARNING py.warnings [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:27:55.378752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: warnings.warn(msg) Aug 24 11:27:55.378752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: Aug 24 11:27:55.380100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:55.534411 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 754/1497] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:27:55 2022] POST /identity/v3/projects => generated 377 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:55.538520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:55.540094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:55.573606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769bef9-855b-4ad8-b304-2af6424f12fe admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:55.668633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:55.669269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:55.669543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:55.670593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:55.671309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:55.672325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:27:55.672325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Aug 24 11:27:55.672325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: Aug 24 11:27:55.673592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:55.845812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 744/1498] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:27:55 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:27:55.849926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:55.851467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:56.032264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edf8af-0436-4a00-8984-a7f0cb2d7e48 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:56.113617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:56.125823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:56.134763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:56.134763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:56.134763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:56.134763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:56.134763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WARNING py.warnings [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:27:56.134763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: warnings.warn(msg) Aug 24 11:27:56.134763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: Aug 24 11:27:56.135409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:56.142013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 755/1499] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:56.146204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:56.147717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:56.159722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d82e1c-acbe-4cc6-b6cf-9fcda9c4cf3f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:56.184936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:56.255226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c7043-5edf-42bb-9758-104c8f007f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:56.255867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c7043-5edf-42bb-9758-104c8f007f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:56.256378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c7043-5edf-42bb-9758-104c8f007f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:56.256743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c7043-5edf-42bb-9758-104c8f007f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:56.257557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c7043-5edf-42bb-9758-104c8f007f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:56.259012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-5e7c7043-5edf-42bb-9758-104c8f007f9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:27:56.259012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Aug 24 11:27:56.259012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: Aug 24 11:27:56.259900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c7043-5edf-42bb-9758-104c8f007f9a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:56.272971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 745/1500] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:27:56.277235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:56.278734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:56.312243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925b7cc-f1a3-48f8-904d-bcfde7bd5c94 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:56.402712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba60da-59fe-4260-8c56-a67d2edee0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:56.403274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba60da-59fe-4260-8c56-a67d2edee0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:56.403546 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba60da-59fe-4260-8c56-a67d2edee0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:56.403815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba60da-59fe-4260-8c56-a67d2edee0ba admin admin] PATH_INFO: `/v3/projects/252e5725e5194c4d8bbfed029d8fec53/users/5dbbf4188f854a5baffb1a6982e0b68c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:56.426188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba60da-59fe-4260-8c56-a67d2edee0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=252e5725e5194c4d8bbfed029d8fec53, user_id=5dbbf4188f854a5baffb1a6982e0b68c, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:56.427260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WARNING py.warnings [None req-7cba60da-59fe-4260-8c56-a67d2edee0ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:27:56.427260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: warnings.warn(msg) Aug 24 11:27:56.427260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: Aug 24 11:27:56.429222 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba60da-59fe-4260-8c56-a67d2edee0ba admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:56.514635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 756/1501] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:27:56 2022] PUT /identity/v3/projects/252e5725e5194c4d8bbfed029d8fec53/users/5dbbf4188f854a5baffb1a6982e0b68c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:27:56.526665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd0f95-a244-429b-80c3-b8ae4f4f2191 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:56.527018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd0f95-a244-429b-80c3-b8ae4f4f2191 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:56.527373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd0f95-a244-429b-80c3-b8ae4f4f2191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:56.583350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-22fd0f95-a244-429b-80c3-b8ae4f4f2191 None None] MFA Rules not processed for user `5dbbf4188f854a5baffb1a6982e0b68c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:56.584538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-22fd0f95-a244-429b-80c3-b8ae4f4f2191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:56.662599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 746/1502] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:27:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:56.676270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2813c-3b4b-4edd-8dc8-e3db14a1f84c None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:56.676551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2813c-3b4b-4edd-8dc8-e3db14a1f84c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:56.676842 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2813c-3b4b-4edd-8dc8-e3db14a1f84c None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:56.678227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 757/1503] 10.222.0.84 () {54 vars in 964 bytes} [Wed Aug 24 11:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:27:56.683012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65ca1-c512-4106-a1b2-4c84e080d0de None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:56.683296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65ca1-c512-4106-a1b2-4c84e080d0de None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:56.683565 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65ca1-c512-4106-a1b2-4c84e080d0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:56.714422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:56.762069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-87c65ca1-c512-4106-a1b2-4c84e080d0de None None] MFA Rules not processed for user `d5ce16d5ea8240debfcb69ac7b1f71c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:56.763282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-87c65ca1-c512-4106-a1b2-4c84e080d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:56.838142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 747/1504] 10.222.0.84 () {58 vars in 1061 bytes} [Wed Aug 24 11:27:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:56.846050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2813c-3b4b-4edd-8dc8-e3db14a1f84c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:56.847607 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fde2813c-3b4b-4edd-8dc8-e3db14a1f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:56.906826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:56.960572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2813c-3b4b-4edd-8dc8-e3db14a1f84c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:57.036817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:57.037527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:57.037731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:57.037996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:57.039585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:57.149037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:57.150464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:57.153958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:57.171254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 758/1505] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:27:57.215641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f9007-d37f-44da-b7dc-6ed1e54ba9ed None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:57.216071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f9007-d37f-44da-b7dc-6ed1e54ba9ed None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:57.216218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f9007-d37f-44da-b7dc-6ed1e54ba9ed None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:57.218129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 748/1506] 10.222.0.84 () {52 vars in 830 bytes} [Wed Aug 24 11:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:27:57.221646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:57.222846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:57.320657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b681e9-c3bc-4d67-9459-5a0233fb49ab service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:57.415241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:57.415860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:57.416132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:57.416394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] PATH_INFO: `/v3/users/5dbbf4188f854a5baffb1a6982e0b68c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:57.428422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dbbf4188f854a5baffb1a6982e0b68c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:57.429540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WARNING py.warnings [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:27:57.429540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: warnings.warn(msg) Aug 24 11:27:57.429540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: Aug 24 11:27:57.430737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:57.440860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:57.708357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dbbf4188f854a5baffb1a6982e0b68c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:57.708809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dbbf4188f854a5baffb1a6982e0b68c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:57.709347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dbbf4188f854a5baffb1a6982e0b68c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:57.724774 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dbbf4188f854a5baffb1a6982e0b68c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:57.747188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dbbf4188f854a5baffb1a6982e0b68c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:57.841675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 759/1507] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:27:57 2022] DELETE /identity/v3/users/5dbbf4188f854a5baffb1a6982e0b68c => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:27:57.846044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4772ea4-333a-4742-975c-b3e811ba9823 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:57.846328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4772ea4-333a-4742-975c-b3e811ba9823 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:57.846602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4772ea4-333a-4742-975c-b3e811ba9823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:57.903786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e4772ea4-333a-4742-975c-b3e811ba9823 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:27:57.905037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e4772ea4-333a-4742-975c-b3e811ba9823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:57.979604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:57.992074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 749/1508] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:27:58.002122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:58.003352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:58.101975 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840542d7-e6f3-4a7f-98fb-e77586b289ec admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:58.158617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:27:58.177519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee2cf3-89c5-4ce7-9382-19eafb740396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:58.178124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee2cf3-89c5-4ce7-9382-19eafb740396 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:58.178400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee2cf3-89c5-4ce7-9382-19eafb740396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:58.178668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee2cf3-89c5-4ce7-9382-19eafb740396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:58.180241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-61ee2cf3-89c5-4ce7-9382-19eafb740396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:58.288340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee2cf3-89c5-4ce7-9382-19eafb740396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:58.289807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee2cf3-89c5-4ce7-9382-19eafb740396 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:58.309005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 760/1509] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:27:58.784975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4772ea4-333a-4742-975c-b3e811ba9823 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:27:58.786610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e4772ea4-333a-4742-975c-b3e811ba9823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:27:58.798572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4772ea4-333a-4742-975c-b3e811ba9823 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:27:58.903373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:27:58.904715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:27:58.905023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:27:58.905465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] PATH_INFO: `/v3/projects/252e5725e5194c4d8bbfed029d8fec53` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:27:58.920887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252e5725e5194c4d8bbfed029d8fec53)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:27:58.921908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:27:58.921908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Aug 24 11:27:58.921908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: Aug 24 11:27:58.923103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:27:59.050108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252e5725e5194c4d8bbfed029d8fec53', 'request_id': 'req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:59.063314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252e5725e5194c4d8bbfed029d8fec53', 'request_id': 'req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:59.150298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252e5725e5194c4d8bbfed029d8fec53', 'request_id': 'req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:59.177013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] The token cache is being invalidate because project 252e5725e5194c4d8bbfed029d8fec53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:27:59.177323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:59.177766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-77dcbc0c-fc8a-467c-9acd-a58bf1b43421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:27:59.179583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 750/1510] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:27:58 2022] DELETE /identity/v3/projects/252e5725e5194c4d8bbfed029d8fec53 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:28:08.306670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=62 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:28:09.174196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:27.577385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ed5d5-54d0-4304-853c-26aa6b21ad33 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:27.577385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ed5d5-54d0-4304-853c-26aa6b21ad33 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:27.577385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ed5d5-54d0-4304-853c-26aa6b21ad33 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:27.579304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:29:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:29:27.587798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfd6ea-d8f3-4ea6-ae36-06917016bbe6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:27.588404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfd6ea-d8f3-4ea6-ae36-06917016bbe6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:27.588636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfd6ea-d8f3-4ea6-ae36-06917016bbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:27.711533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d6dfd6ea-d8f3-4ea6-ae36-06917016bbe6 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:27.712769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d6dfd6ea-d8f3-4ea6-ae36-06917016bbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:27.848278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:28.012928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ed5d5-54d0-4304-853c-26aa6b21ad33 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:28.015723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a62ed5d5-54d0-4304-853c-26aa6b21ad33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:28.187510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ed5d5-54d0-4304-853c-26aa6b21ad33 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:28.194245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ef285-09f9-48e6-94de-15b46b90c534 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:28.195017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ef285-09f9-48e6-94de-15b46b90c534 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:28.195294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ef285-09f9-48e6-94de-15b46b90c534 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:28.195562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ef285-09f9-48e6-94de-15b46b90c534 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:28.205396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ef285-09f9-48e6-94de-15b46b90c534 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:28.207880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ef285-09f9-48e6-94de-15b46b90c534 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:28.221697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 762/1513] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:29.142978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486370d7-056f-416d-afe0-8a80cc43a890 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:29.143569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486370d7-056f-416d-afe0-8a80cc43a890 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:29.143734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486370d7-056f-416d-afe0-8a80cc43a890 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:29.145281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:29:29.151504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc097e1a-f9d1-4674-8a08-552f9bea0c13 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:29.152238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc097e1a-f9d1-4674-8a08-552f9bea0c13 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:29.152238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc097e1a-f9d1-4674-8a08-552f9bea0c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:29.181015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cc097e1a-f9d1-4674-8a08-552f9bea0c13 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:29.182914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cc097e1a-f9d1-4674-8a08-552f9bea0c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:29.195156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:29:29.357651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486370d7-056f-416d-afe0-8a80cc43a890 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:29.376383 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486370d7-056f-416d-afe0-8a80cc43a890 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:29.384126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffd415-8097-4a8e-81e6-60a75c8d2961 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:29.384743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffd415-8097-4a8e-81e6-60a75c8d2961 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:29.385053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffd415-8097-4a8e-81e6-60a75c8d2961 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:29.385325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffd415-8097-4a8e-81e6-60a75c8d2961 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:29.394644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffd415-8097-4a8e-81e6-60a75c8d2961 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:29.396243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffd415-8097-4a8e-81e6-60a75c8d2961 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:29.406807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 753/1516] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:30.057449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8a176-625e-49c5-a4e0-0cad28e5ee3b None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:30.057719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8a176-625e-49c5-a4e0-0cad28e5ee3b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:30.057992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8a176-625e-49c5-a4e0-0cad28e5ee3b None None] PATH_INFO: `/v3` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:30.059295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 764/1517] 10.222.0.94 () {54 vars in 925 bytes} [Wed Aug 24 11:29:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:29:30.069074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1076e09-108c-460a-8b8a-529eaf6dd1d1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:30.069517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1076e09-108c-460a-8b8a-529eaf6dd1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:30.069975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1076e09-108c-460a-8b8a-529eaf6dd1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:30.147303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f1076e09-108c-460a-8b8a-529eaf6dd1d1 None None] MFA Rules not processed for user `c7cc35f5ddf0446e90ded5c85f21f37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:30.148539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f1076e09-108c-460a-8b8a-529eaf6dd1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:30.168917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:30.236726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 754/1518] 10.222.0.94 () {58 vars in 1015 bytes} [Wed Aug 24 11:29:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:39.173086 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=56 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:39.894554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b7716-39a0-4ff6-acc8-e23f96c3d917 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:39.894760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b7716-39a0-4ff6-acc8-e23f96c3d917 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:39.895002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b7716-39a0-4ff6-acc8-e23f96c3d917 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:39.896344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 765/1519] 10.222.0.84 () {54 vars in 961 bytes} [Wed Aug 24 11:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:29:39.902019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2cef1-0769-4a0f-9bc9-b76979ae0a65 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:39.902547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2cef1-0769-4a0f-9bc9-b76979ae0a65 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:39.902547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2cef1-0769-4a0f-9bc9-b76979ae0a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:39.923657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-09f2cef1-0769-4a0f-9bc9-b76979ae0a65 None None] MFA Rules not processed for user `c7cc35f5ddf0446e90ded5c85f21f37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:39.925575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-09f2cef1-0769-4a0f-9bc9-b76979ae0a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:39.934560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 755/1520] 10.222.0.84 () {58 vars in 1058 bytes} [Wed Aug 24 11:29:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:29:39.945973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b7716-39a0-4ff6-acc8-e23f96c3d917 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:39.959125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b7716-39a0-4ff6-acc8-e23f96c3d917 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:39.964223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18854762-4571-4089-b260-26d68252c17c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:39.964843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18854762-4571-4089-b260-26d68252c17c service placement] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:39.965114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18854762-4571-4089-b260-26d68252c17c service placement] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:39.965379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18854762-4571-4089-b260-26d68252c17c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:39.974583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18854762-4571-4089-b260-26d68252c17c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:39.976152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18854762-4571-4089-b260-26d68252c17c service placement] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:39.985867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 766/1521] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:40.676566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96aade-4737-4a95-ae0c-b6199ffcf986 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:40.676957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96aade-4737-4a95-ae0c-b6199ffcf986 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:40.677306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96aade-4737-4a95-ae0c-b6199ffcf986 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:40.678727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:29:40.683770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670194ae-5528-4a48-a719-19b222667bea None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:40.684317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670194ae-5528-4a48-a719-19b222667bea None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:40.684317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670194ae-5528-4a48-a719-19b222667bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:40.704125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-670194ae-5528-4a48-a719-19b222667bea None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:40.705344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-670194ae-5528-4a48-a719-19b222667bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:40.713127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:40.873783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96aade-4737-4a95-ae0c-b6199ffcf986 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:40.887903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96aade-4737-4a95-ae0c-b6199ffcf986 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:40.893602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debc4b2-7957-4f57-9649-15f951b8c00f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:40.894149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debc4b2-7957-4f57-9649-15f951b8c00f service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:40.894417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debc4b2-7957-4f57-9649-15f951b8c00f service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:40.894685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debc4b2-7957-4f57-9649-15f951b8c00f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:40.902909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debc4b2-7957-4f57-9649-15f951b8c00f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:40.904424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debc4b2-7957-4f57-9649-15f951b8c00f service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:40.914637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 757/1524] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:41.738768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bee27b-781b-488f-b9a7-4231e9340fac None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:41.739358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bee27b-781b-488f-b9a7-4231e9340fac None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:41.739358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bee27b-781b-488f-b9a7-4231e9340fac None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:41.740643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 11:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:29:41.745619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604e13c-cbc0-40b4-a91b-b36c1e4b2025 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:41.745903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604e13c-cbc0-40b4-a91b-b36c1e4b2025 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:41.746178 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604e13c-cbc0-40b4-a91b-b36c1e4b2025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:41.766563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6604e13c-cbc0-40b4-a91b-b36c1e4b2025 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:41.767821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6604e13c-cbc0-40b4-a91b-b36c1e4b2025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:41.776261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 11:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:41.932709 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bee27b-781b-488f-b9a7-4231e9340fac None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:41.948306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bee27b-781b-488f-b9a7-4231e9340fac None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:41.953724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1ee86-10a4-44bf-90a1-0b714f67532a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:41.954320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1ee86-10a4-44bf-90a1-0b714f67532a service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:41.954595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1ee86-10a4-44bf-90a1-0b714f67532a service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:41.954863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1ee86-10a4-44bf-90a1-0b714f67532a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:41.963328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1ee86-10a4-44bf-90a1-0b714f67532a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:41.964951 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1ee86-10a4-44bf-90a1-0b714f67532a service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:41.974186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 769/1527] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:46.121374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8b96a-cde6-4721-a947-9a58d65fd271 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:46.121963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8b96a-cde6-4721-a947-9a58d65fd271 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:46.121963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8b96a-cde6-4721-a947-9a58d65fd271 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:46.123257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:29:46.129085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50cf3f-1f3c-4f99-b076-5147c0f41c28 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:46.129607 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50cf3f-1f3c-4f99-b076-5147c0f41c28 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:46.129607 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50cf3f-1f3c-4f99-b076-5147c0f41c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:46.152053 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9f50cf3f-1f3c-4f99-b076-5147c0f41c28 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:46.153292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9f50cf3f-1f3c-4f99-b076-5147c0f41c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:46.161733 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:46.458640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8b96a-cde6-4721-a947-9a58d65fd271 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:46.460362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4ed8b96a-cde6-4721-a947-9a58d65fd271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:46.540900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8b96a-cde6-4721-a947-9a58d65fd271 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:46.544434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04e4e4-7da5-4b60-9e01-5c7d29c97d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:46.545059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04e4e4-7da5-4b60-9e01-5c7d29c97d2f service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:46.545333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04e4e4-7da5-4b60-9e01-5c7d29c97d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:46.545596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04e4e4-7da5-4b60-9e01-5c7d29c97d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:46.555242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04e4e4-7da5-4b60-9e01-5c7d29c97d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:46.556763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04e4e4-7da5-4b60-9e01-5c7d29c97d2f service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:46.567019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 760/1530] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:48.489280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b4e33-fa2b-4b5e-a4b9-98c28336ada3 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:48.489852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b4e33-fa2b-4b5e-a4b9-98c28336ada3 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:48.489852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b4e33-fa2b-4b5e-a4b9-98c28336ada3 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:48.491179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:29:48.496535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5026dc-6e40-46a0-840f-2b8e676add1b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:48.496960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5026dc-6e40-46a0-840f-2b8e676add1b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:48.497104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5026dc-6e40-46a0-840f-2b8e676add1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:48.516657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ed5026dc-6e40-46a0-840f-2b8e676add1b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:48.517969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ed5026dc-6e40-46a0-840f-2b8e676add1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:48.526413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:48.812113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b4e33-fa2b-4b5e-a4b9-98c28336ada3 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:48.823775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b4e33-fa2b-4b5e-a4b9-98c28336ada3 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:48.827199 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4871f-5504-4ec6-a558-7d3e9643d07b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:48.827781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4871f-5504-4ec6-a558-7d3e9643d07b service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:48.828053 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4871f-5504-4ec6-a558-7d3e9643d07b service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:48.828305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4871f-5504-4ec6-a558-7d3e9643d07b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:48.837156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4871f-5504-4ec6-a558-7d3e9643d07b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:48.838561 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4871f-5504-4ec6-a558-7d3e9643d07b service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:48.848211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 772/1533] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:52.587616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac539abd-4387-4aac-a490-60bc3257f232 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:52.588198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac539abd-4387-4aac-a490-60bc3257f232 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:52.588198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac539abd-4387-4aac-a490-60bc3257f232 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:52.589540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:29:52.595094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636c88-2890-4b16-bb78-e1187d75bd71 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:52.595553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636c88-2890-4b16-bb78-e1187d75bd71 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:52.595659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636c88-2890-4b16-bb78-e1187d75bd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:52.615901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9a636c88-2890-4b16-bb78-e1187d75bd71 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:52.617484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9a636c88-2890-4b16-bb78-e1187d75bd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:52.625310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:53.180236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac539abd-4387-4aac-a490-60bc3257f232 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:53.181970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ac539abd-4387-4aac-a490-60bc3257f232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:53.247545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:29:53.261063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac539abd-4387-4aac-a490-60bc3257f232 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:53.267428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73698372-dac3-4ada-92ee-ebe350fc1535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:53.267428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73698372-dac3-4ada-92ee-ebe350fc1535 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:53.267428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73698372-dac3-4ada-92ee-ebe350fc1535 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:53.267428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73698372-dac3-4ada-92ee-ebe350fc1535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:53.275097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73698372-dac3-4ada-92ee-ebe350fc1535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:53.276535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73698372-dac3-4ada-92ee-ebe350fc1535 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:53.286372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 763/1536] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:29:55.218174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab76d82-e30d-460d-8b5a-68fcd0aea05d None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:55.218719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab76d82-e30d-460d-8b5a-68fcd0aea05d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:55.218719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab76d82-e30d-460d-8b5a-68fcd0aea05d None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:55.220071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 11:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:29:55.225582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a433b8-5158-43c0-befb-59b7144468ea None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:55.226003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a433b8-5158-43c0-befb-59b7144468ea None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:55.226117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a433b8-5158-43c0-befb-59b7144468ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:55.245598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-49a433b8-5158-43c0-befb-59b7144468ea None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:29:55.246884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-49a433b8-5158-43c0-befb-59b7144468ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:55.255408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 11:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:29:55.805519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab76d82-e30d-460d-8b5a-68fcd0aea05d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:29:55.807163 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3ab76d82-e30d-460d-8b5a-68fcd0aea05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:29:55.820235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab76d82-e30d-460d-8b5a-68fcd0aea05d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:29:55.823663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb7455-c082-4a55-8e11-02712b06e1c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:29:55.824249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7455-c082-4a55-8e11-02712b06e1c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:29:55.824518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7455-c082-4a55-8e11-02712b06e1c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:29:55.824779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7455-c082-4a55-8e11-02712b06e1c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:29:55.833909 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb7455-c082-4a55-8e11-02712b06e1c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:29:55.835306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb7455-c082-4a55-8e11-02712b06e1c6 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:29:55.844373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 775/1539] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:30:05.240553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=11 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:30:05.840380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=33 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:22.454760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863436f1-f0a7-4127-bb11-d0c7d30109a4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.457045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863436f1-f0a7-4127-bb11-d0c7d30109a4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.457045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863436f1-f0a7-4127-bb11-d0c7d30109a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.474234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438636cf-6b5e-4a15-a2a2-e8b201333045 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.475262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438636cf-6b5e-4a15-a2a2-e8b201333045 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.475738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438636cf-6b5e-4a15-a2a2-e8b201333045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.493127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-863436f1-f0a7-4127-bb11-d0c7d30109a4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:22.494599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-863436f1-f0a7-4127-bb11-d0c7d30109a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:22.498907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-438636cf-6b5e-4a15-a2a2-e8b201333045 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:22.500477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-438636cf-6b5e-4a15-a2a2-e8b201333045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:22.515587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 765/1540] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:22.524592 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 776/1541] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:31:22.539703 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01369f18-a9f9-454e-a01a-208ddd4f7c78 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.540356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01369f18-a9f9-454e-a01a-208ddd4f7c78 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.540882 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01369f18-a9f9-454e-a01a-208ddd4f7c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.547376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc167ac-8f17-4b8e-ba2f-5fa1078b1917 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.547894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc167ac-8f17-4b8e-ba2f-5fa1078b1917 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.548357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc167ac-8f17-4b8e-ba2f-5fa1078b1917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.573895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-01369f18-a9f9-454e-a01a-208ddd4f7c78 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:22.575804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-01369f18-a9f9-454e-a01a-208ddd4f7c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:22.578072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6fc167ac-8f17-4b8e-ba2f-5fa1078b1917 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:22.579407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6fc167ac-8f17-4b8e-ba2f-5fa1078b1917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:22.594574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 766/1542] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:31:22.597653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 777/1543] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:22.604136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01369f18-a9f9-454e-a01a-208ddd4f7c78 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:22.611207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc167ac-8f17-4b8e-ba2f-5fa1078b1917 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:22.620302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01369f18-a9f9-454e-a01a-208ddd4f7c78 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:22.624366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc167ac-8f17-4b8e-ba2f-5fa1078b1917 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:22.631559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eea4a3-0d13-43dc-9c67-bc4ad67d1daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:22.632674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eea4a3-0d13-43dc-9c67-bc4ad67d1daf admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.633127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eea4a3-0d13-43dc-9c67-bc4ad67d1daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.633651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eea4a3-0d13-43dc-9c67-bc4ad67d1daf admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.634330 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f89e2-eea2-42c6-8443-e85383c23f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:22.634819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eea4a3-0d13-43dc-9c67-bc4ad67d1daf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:22.635188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f89e2-eea2-42c6-8443-e85383c23f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.635447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eea4a3-0d13-43dc-9c67-bc4ad67d1daf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:22.635639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f89e2-eea2-42c6-8443-e85383c23f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.636013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f89e2-eea2-42c6-8443-e85383c23f89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.636751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f89e2-eea2-42c6-8443-e85383c23f89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:22.637172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f89e2-eea2-42c6-8443-e85383c23f89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:22.638068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eea4a3-0d13-43dc-9c67-bc4ad67d1daf admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:22.644090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WARNING py.warnings [None req-471f89e2-eea2-42c6-8443-e85383c23f89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:31:22.644090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: warnings.warn(msg) Aug 24 11:31:22.644090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: Aug 24 11:31:22.644090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f89e2-eea2-42c6-8443-e85383c23f89 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:22.668119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 778/1544] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:31:22.668642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 767/1545] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:31:22.669685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ca9c7-650e-4765-a057-82bc638d7fe6 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.670080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605513-3ff9-4806-8834-50e7bc7a05b0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.670210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ca9c7-650e-4765-a057-82bc638d7fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.670362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605513-3ff9-4806-8834-50e7bc7a05b0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.670597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ca9c7-650e-4765-a057-82bc638d7fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.670714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605513-3ff9-4806-8834-50e7bc7a05b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.706975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-56605513-3ff9-4806-8834-50e7bc7a05b0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:22.708090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-56605513-3ff9-4806-8834-50e7bc7a05b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:22.717883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6d7ca9c7-650e-4765-a057-82bc638d7fe6 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:22.719191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6d7ca9c7-650e-4765-a057-82bc638d7fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:22.723439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 768/1546] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:22.724953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0775f14-e039-4bb7-a5f6-eecb7bbd2a17 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.725331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0775f14-e039-4bb7-a5f6-eecb7bbd2a17 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.725877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0775f14-e039-4bb7-a5f6-eecb7bbd2a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.737397 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 779/1547] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:22.738007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ca9c7-650e-4765-a057-82bc638d7fe6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:22.748135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ca9c7-650e-4765-a057-82bc638d7fe6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:22.757828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b0775f14-e039-4bb7-a5f6-eecb7bbd2a17 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:22.761960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b0775f14-e039-4bb7-a5f6-eecb7bbd2a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:22.769585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c04174-ecd4-4aa5-9322-e84d05eacdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:22.769797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 769/1548] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:22.770218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c04174-ecd4-4aa5-9322-e84d05eacdbc admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.770500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c04174-ecd4-4aa5-9322-e84d05eacdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.770752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0775f14-e039-4bb7-a5f6-eecb7bbd2a17 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:22.770861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c04174-ecd4-4aa5-9322-e84d05eacdbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.771477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c04174-ecd4-4aa5-9322-e84d05eacdbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:22.773496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c04174-ecd4-4aa5-9322-e84d05eacdbc admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:22.782060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0775f14-e039-4bb7-a5f6-eecb7bbd2a17 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:22.787452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae7ab6-056f-41a2-ab8a-b58fbbcfaeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:22.788154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7ab6-056f-41a2-ab8a-b58fbbcfaeef admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.788494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7ab6-056f-41a2-ab8a-b58fbbcfaeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.788858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae7ab6-056f-41a2-ab8a-b58fbbcfaeef admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.789550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae7ab6-056f-41a2-ab8a-b58fbbcfaeef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:22.790765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-97ae7ab6-056f-41a2-ab8a-b58fbbcfaeef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:31:22.790765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Aug 24 11:31:22.790765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: Aug 24 11:31:22.792094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae7ab6-056f-41a2-ab8a-b58fbbcfaeef admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:22.897076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 780/1549] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/projects => generated 391 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:22.898388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61e899-af44-4b3e-b91f-7bee18404222 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.898668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61e899-af44-4b3e-b91f-7bee18404222 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.898939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61e899-af44-4b3e-b91f-7bee18404222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.917937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-5e61e899-af44-4b3e-b91f-7bee18404222 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:22.919140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5e61e899-af44-4b3e-b91f-7bee18404222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:22.926380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 770/1550] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:22.927258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3674f93-ce41-4ba9-aa90-37ecbcaa5317 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.927546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3674f93-ce41-4ba9-aa90-37ecbcaa5317 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.927878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3674f93-ce41-4ba9-aa90-37ecbcaa5317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:22.949295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c3674f93-ce41-4ba9-aa90-37ecbcaa5317 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:22.950582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c3674f93-ce41-4ba9-aa90-37ecbcaa5317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:22.982066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 781/1551] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:22.983237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22611609-ed78-44f8-9cc6-5d55305b5fa7 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:22.983521 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22611609-ed78-44f8-9cc6-5d55305b5fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:22.983792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22611609-ed78-44f8-9cc6-5d55305b5fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.003981 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-22611609-ed78-44f8-9cc6-5d55305b5fa7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:23.005248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-22611609-ed78-44f8-9cc6-5d55305b5fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:23.011968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 771/1552] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:23.013428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 782/1553] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:23.013556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d232f31-6396-47b2-a0b3-16b182e722e7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.013741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d232f31-6396-47b2-a0b3-16b182e722e7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.014064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d232f31-6396-47b2-a0b3-16b182e722e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.018769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365b97a-d861-4aaf-8b00-2529d9f573ba None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.019053 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365b97a-d861-4aaf-8b00-2529d9f573ba None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.019326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365b97a-d861-4aaf-8b00-2529d9f573ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.038017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0d232f31-6396-47b2-a0b3-16b182e722e7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:23.039391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0d232f31-6396-47b2-a0b3-16b182e722e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:23.042280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-1365b97a-d861-4aaf-8b00-2529d9f573ba None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:23.043445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1365b97a-d861-4aaf-8b00-2529d9f573ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:23.047392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 772/1554] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:23.049256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad68f07-0174-4da2-be66-addfec819eb1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.049826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad68f07-0174-4da2-be66-addfec819eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.050099 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad68f07-0174-4da2-be66-addfec819eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.058975 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 783/1555] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:23.059778 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365b97a-d861-4aaf-8b00-2529d9f573ba None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.070917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365b97a-d861-4aaf-8b00-2529d9f573ba None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.075342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9ad68f07-0174-4da2-be66-addfec819eb1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:23.075515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652494b-532f-4a0a-926b-d2e2e7d0ae1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.076113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652494b-532f-4a0a-926b-d2e2e7d0ae1c admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.076391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652494b-532f-4a0a-926b-d2e2e7d0ae1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.076534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9ad68f07-0174-4da2-be66-addfec819eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:23.076647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652494b-532f-4a0a-926b-d2e2e7d0ae1c admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.077417 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652494b-532f-4a0a-926b-d2e2e7d0ae1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.078359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WARNING py.warnings [None req-e652494b-532f-4a0a-926b-d2e2e7d0ae1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:31:23.078359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: warnings.warn(msg) Aug 24 11:31:23.078359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: Aug 24 11:31:23.079599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652494b-532f-4a0a-926b-d2e2e7d0ae1c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.084645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 773/1556] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:23.085437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad68f07-0174-4da2-be66-addfec819eb1 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.095560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad68f07-0174-4da2-be66-addfec819eb1 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.102472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5c375-6fa8-45d8-a859-ca5e005c8257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.103983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5c375-6fa8-45d8-a859-ca5e005c8257 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.103983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5c375-6fa8-45d8-a859-ca5e005c8257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.103983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5c375-6fa8-45d8-a859-ca5e005c8257 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.104375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5c375-6fa8-45d8-a859-ca5e005c8257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.106348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5c375-6fa8-45d8-a859-ca5e005c8257 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.196380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 784/1557] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:23.196380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652494b-532f-4a0a-926b-d2e2e7d0ae1c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.203259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652494b-532f-4a0a-926b-d2e2e7d0ae1c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.207423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 774/1558] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:23.207423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5c375-6fa8-45d8-a859-ca5e005c8257 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.208184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70939639-1082-4eea-93d2-eee6bc999fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.208784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70939639-1082-4eea-93d2-eee6bc999fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.209088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70939639-1082-4eea-93d2-eee6bc999fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.209353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70939639-1082-4eea-93d2-eee6bc999fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.209988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70939639-1082-4eea-93d2-eee6bc999fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:23.210285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70939639-1082-4eea-93d2-eee6bc999fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.211883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70939639-1082-4eea-93d2-eee6bc999fd9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.217118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5c375-6fa8-45d8-a859-ca5e005c8257 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.221936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c2077-0e68-4a5c-9821-bfdd943842ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.223901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2077-0e68-4a5c-9821-bfdd943842ea admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.223901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2077-0e68-4a5c-9821-bfdd943842ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.223901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2077-0e68-4a5c-9821-bfdd943842ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.223901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c2077-0e68-4a5c-9821-bfdd943842ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:23.223901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c2077-0e68-4a5c-9821-bfdd943842ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.224857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 785/1559] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:31:23.225652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70939639-1082-4eea-93d2-eee6bc999fd9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.225766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c2077-0e68-4a5c-9821-bfdd943842ea admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.237733 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70939639-1082-4eea-93d2-eee6bc999fd9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.241993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 775/1560] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:31:23.242821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51b238-359e-46e6-aa9a-31802fdcf378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.243665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e14d6a-d108-427f-a0c0-0ad88eb8f0cf None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.243665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e14d6a-d108-427f-a0c0-0ad88eb8f0cf None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.243844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51b238-359e-46e6-aa9a-31802fdcf378 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.243928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e14d6a-d108-427f-a0c0-0ad88eb8f0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.244094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51b238-359e-46e6-aa9a-31802fdcf378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.244359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51b238-359e-46e6-aa9a-31802fdcf378 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.245025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51b238-359e-46e6-aa9a-31802fdcf378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:23.245325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51b238-359e-46e6-aa9a-31802fdcf378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.246901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51b238-359e-46e6-aa9a-31802fdcf378 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.260279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 786/1561] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:31:23.261520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61537-ee05-44ab-880f-d7e0643873c2 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.261796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61537-ee05-44ab-880f-d7e0643873c2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.262064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff61537-ee05-44ab-880f-d7e0643873c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.263471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-89e14d6a-d108-427f-a0c0-0ad88eb8f0cf None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:23.264668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-89e14d6a-d108-427f-a0c0-0ad88eb8f0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:23.272892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 776/1562] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:23.274081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04442d87-0494-4155-87c2-cea51b1ae89e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.274360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04442d87-0494-4155-87c2-cea51b1ae89e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.274636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04442d87-0494-4155-87c2-cea51b1ae89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.281921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6ff61537-ee05-44ab-880f-d7e0643873c2 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:23.283198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6ff61537-ee05-44ab-880f-d7e0643873c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:23.291070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 787/1563] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:23.291850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff61537-ee05-44ab-880f-d7e0643873c2 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.293868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-04442d87-0494-4155-87c2-cea51b1ae89e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:23.295369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-04442d87-0494-4155-87c2-cea51b1ae89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:23.301636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff61537-ee05-44ab-880f-d7e0643873c2 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.304133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 777/1564] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:23.304930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04442d87-0494-4155-87c2-cea51b1ae89e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.306344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaef66a-82f0-434c-b594-0281dfe1e24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.306991 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaef66a-82f0-434c-b594-0281dfe1e24c admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.307331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaef66a-82f0-434c-b594-0281dfe1e24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.307784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaef66a-82f0-434c-b594-0281dfe1e24c admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.308411 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaef66a-82f0-434c-b594-0281dfe1e24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.310120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaef66a-82f0-434c-b594-0281dfe1e24c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.315418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04442d87-0494-4155-87c2-cea51b1ae89e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.320469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28eec5-ae43-4a72-932e-00db0a706a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.321091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28eec5-ae43-4a72-932e-00db0a706a50 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.321320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 788/1565] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:23.321422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28eec5-ae43-4a72-932e-00db0a706a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.321627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28eec5-ae43-4a72-932e-00db0a706a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.322262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28eec5-ae43-4a72-932e-00db0a706a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.322375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaef66a-82f0-434c-b594-0281dfe1e24c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.323835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28eec5-ae43-4a72-932e-00db0a706a50 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.332324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaef66a-82f0-434c-b594-0281dfe1e24c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.337170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 778/1566] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:23.337170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28eec5-ae43-4a72-932e-00db0a706a50 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.337429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d405f-ce47-40c0-8e14-43a72216604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.337703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d405f-ce47-40c0-8e14-43a72216604d admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.337980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d405f-ce47-40c0-8e14-43a72216604d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.338249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d405f-ce47-40c0-8e14-43a72216604d admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.338856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d405f-ce47-40c0-8e14-43a72216604d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.340722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d405f-ce47-40c0-8e14-43a72216604d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.344285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28eec5-ae43-4a72-932e-00db0a706a50 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.348851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5623e-0df9-465d-a1ff-9fc98f5a4b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.349429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5623e-0df9-465d-a1ff-9fc98f5a4b0d admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.349707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5623e-0df9-465d-a1ff-9fc98f5a4b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.349978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5623e-0df9-465d-a1ff-9fc98f5a4b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.350583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5623e-0df9-465d-a1ff-9fc98f5a4b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.352453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5623e-0df9-465d-a1ff-9fc98f5a4b0d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.452195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:23.458806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 789/1567] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:23.465786 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d405f-ce47-40c0-8e14-43a72216604d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.475229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:23.481935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 779/1568] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:23.482730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5623e-0df9-465d-a1ff-9fc98f5a4b0d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.521361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d405f-ce47-40c0-8e14-43a72216604d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.546640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5623e-0df9-465d-a1ff-9fc98f5a4b0d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.551946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f034ebd-c684-4b1a-ab30-a8549823e3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.552609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f034ebd-c684-4b1a-ab30-a8549823e3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.552853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f034ebd-c684-4b1a-ab30-a8549823e3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.553129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f034ebd-c684-4b1a-ab30-a8549823e3ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.553771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f034ebd-c684-4b1a-ab30-a8549823e3ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:23.554067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f034ebd-c684-4b1a-ab30-a8549823e3ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.555659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f034ebd-c684-4b1a-ab30-a8549823e3ad admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.567240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 780/1569] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:31:23.568057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f034ebd-c684-4b1a-ab30-a8549823e3ad admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.577150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd4b72-c294-41ab-9ba7-6da6370eca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.579383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd4b72-c294-41ab-9ba7-6da6370eca3e admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.579383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd4b72-c294-41ab-9ba7-6da6370eca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.579383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd4b72-c294-41ab-9ba7-6da6370eca3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.579383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd4b72-c294-41ab-9ba7-6da6370eca3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.579863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f034ebd-c684-4b1a-ab30-a8549823e3ad admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.581334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd4b72-c294-41ab-9ba7-6da6370eca3e admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.584359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5de7a7-c6b2-41dd-8fae-0906a94e9a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.585004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5de7a7-c6b2-41dd-8fae-0906a94e9a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.585281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5de7a7-c6b2-41dd-8fae-0906a94e9a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.585551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5de7a7-c6b2-41dd-8fae-0906a94e9a21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.586182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5de7a7-c6b2-41dd-8fae-0906a94e9a21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:23.586483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5de7a7-c6b2-41dd-8fae-0906a94e9a21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.588052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5de7a7-c6b2-41dd-8fae-0906a94e9a21 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.601192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 781/1570] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:31:23.602006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5de7a7-c6b2-41dd-8fae-0906a94e9a21 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.612576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5de7a7-c6b2-41dd-8fae-0906a94e9a21 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.619630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8994c-7f35-49ea-aeef-73ba6a9009ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.619630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8994c-7f35-49ea-aeef-73ba6a9009ba admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.619630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8994c-7f35-49ea-aeef-73ba6a9009ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.619630 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8994c-7f35-49ea-aeef-73ba6a9009ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.619997 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8994c-7f35-49ea-aeef-73ba6a9009ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:23.619997 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8994c-7f35-49ea-aeef-73ba6a9009ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.621169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8994c-7f35-49ea-aeef-73ba6a9009ba admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.632822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 782/1571] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:31:23.633599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8994c-7f35-49ea-aeef-73ba6a9009ba admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.643021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8994c-7f35-49ea-aeef-73ba6a9009ba admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.647669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054b8fd-e272-41da-9f4f-34f0f8355860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.649243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054b8fd-e272-41da-9f4f-34f0f8355860 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.649243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054b8fd-e272-41da-9f4f-34f0f8355860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.649243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054b8fd-e272-41da-9f4f-34f0f8355860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.649520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054b8fd-e272-41da-9f4f-34f0f8355860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.651052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054b8fd-e272-41da-9f4f-34f0f8355860 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.662181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 783/1572] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:23.662995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054b8fd-e272-41da-9f4f-34f0f8355860 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.672627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054b8fd-e272-41da-9f4f-34f0f8355860 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.677838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe45b0-0de6-4aa9-a499-2728af5b1fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.678616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe45b0-0de6-4aa9-a499-2728af5b1fbc admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.683582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe45b0-0de6-4aa9-a499-2728af5b1fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.683582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe45b0-0de6-4aa9-a499-2728af5b1fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.683582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe45b0-0de6-4aa9-a499-2728af5b1fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.683582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe45b0-0de6-4aa9-a499-2728af5b1fbc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.692193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 784/1573] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:23.693125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe45b0-0de6-4aa9-a499-2728af5b1fbc admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.702922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe45b0-0de6-4aa9-a499-2728af5b1fbc admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.707770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d798-07d4-452f-b879-833c2ff94649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.708337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d798-07d4-452f-b879-833c2ff94649 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.708613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d798-07d4-452f-b879-833c2ff94649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.708918 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8d798-07d4-452f-b879-833c2ff94649 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.709519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8d798-07d4-452f-b879-833c2ff94649 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.711387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8d798-07d4-452f-b879-833c2ff94649 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.810422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 785/1574] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:23.811257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d798-07d4-452f-b879-833c2ff94649 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.818526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 790/1575] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/projects => generated 357 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:23.819188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd4b72-c294-41ab-9ba7-6da6370eca3e admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.824083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8d798-07d4-452f-b879-833c2ff94649 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.881739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd4b72-c294-41ab-9ba7-6da6370eca3e admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.882713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15603fe7-c51b-4b18-8413-a99626a2e4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.883319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15603fe7-c51b-4b18-8413-a99626a2e4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.883592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15603fe7-c51b-4b18-8413-a99626a2e4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.883862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15603fe7-c51b-4b18-8413-a99626a2e4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.884470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15603fe7-c51b-4b18-8413-a99626a2e4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.886439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19db1b-3488-46a0-a9ea-984dc82dc9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.886586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15603fe7-c51b-4b18-8413-a99626a2e4a2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.887036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19db1b-3488-46a0-a9ea-984dc82dc9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.887310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19db1b-3488-46a0-a9ea-984dc82dc9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.887601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19db1b-3488-46a0-a9ea-984dc82dc9b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:23.888338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19db1b-3488-46a0-a9ea-984dc82dc9b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:23.890279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19db1b-3488-46a0-a9ea-984dc82dc9b5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:23.982571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 786/1576] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:23.983371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15603fe7-c51b-4b18-8413-a99626a2e4a2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:23.992825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15603fe7-c51b-4b18-8413-a99626a2e4a2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:23.997600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4381b3-1044-4201-9c30-57916bbddc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:23.998786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4381b3-1044-4201-9c30-57916bbddc0c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:23.999100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4381b3-1044-4201-9c30-57916bbddc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:23.999379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4381b3-1044-4201-9c30-57916bbddc0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.000000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4381b3-1044-4201-9c30-57916bbddc0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.001983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4381b3-1044-4201-9c30-57916bbddc0c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.012638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 791/1577] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:24.012638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19db1b-3488-46a0-a9ea-984dc82dc9b5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.020736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:24.034574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:24.075070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19db1b-3488-46a0-a9ea-984dc82dc9b5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.079675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b072522-0a6a-4fac-ba54-0de854eb7404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.080856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b072522-0a6a-4fac-ba54-0de854eb7404 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.081235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b072522-0a6a-4fac-ba54-0de854eb7404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.081507 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b072522-0a6a-4fac-ba54-0de854eb7404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.082185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b072522-0a6a-4fac-ba54-0de854eb7404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.084066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b072522-0a6a-4fac-ba54-0de854eb7404 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.133487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 787/1578] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:23 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:24.133487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4381b3-1044-4201-9c30-57916bbddc0c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.193401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4381b3-1044-4201-9c30-57916bbddc0c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.198009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a1872-82ad-454f-bf2d-fb84c0ba386c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.198666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a1872-82ad-454f-bf2d-fb84c0ba386c admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.198942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a1872-82ad-454f-bf2d-fb84c0ba386c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.199212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a1872-82ad-454f-bf2d-fb84c0ba386c admin admin] PATH_INFO: `/v3/projects/048c7f026f304084960d77bf46f367d8/users/2d5bfffc316f481183093e386a326946/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.204050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 792/1579] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:24 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:24.204050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b072522-0a6a-4fac-ba54-0de854eb7404 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.218252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a1872-82ad-454f-bf2d-fb84c0ba386c admin admin] RBAC: Authorizing `identity:create_grant(project_id=048c7f026f304084960d77bf46f367d8, user_id=2d5bfffc316f481183093e386a326946, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.219302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-f30a1872-82ad-454f-bf2d-fb84c0ba386c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:31:24.219302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Aug 24 11:31:24.219302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: Aug 24 11:31:24.221738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a1872-82ad-454f-bf2d-fb84c0ba386c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.264120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b072522-0a6a-4fac-ba54-0de854eb7404 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.269460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af64ec-38d1-41a9-b0ea-fc803d30bac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.270140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af64ec-38d1-41a9-b0ea-fc803d30bac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.270455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af64ec-38d1-41a9-b0ea-fc803d30bac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.270727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af64ec-38d1-41a9-b0ea-fc803d30bac1 admin admin] PATH_INFO: `/v3/projects/099fe6c7868e4c2990813036624f4cd8/users/3dd192e05b7b42209a28ee1419827a6d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.283638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 788/1580] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:24 2022] PUT /identity/v3/projects/048c7f026f304084960d77bf46f367d8/users/2d5bfffc316f481183093e386a326946/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:24.284726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a1872-82ad-454f-bf2d-fb84c0ba386c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.286767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af64ec-38d1-41a9-b0ea-fc803d30bac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=099fe6c7868e4c2990813036624f4cd8, user_id=3dd192e05b7b42209a28ee1419827a6d, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.290013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af64ec-38d1-41a9-b0ea-fc803d30bac1 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.344941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a1872-82ad-454f-bf2d-fb84c0ba386c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.350121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfad60-6db1-4286-96d0-7c09df0928a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.350724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfad60-6db1-4286-96d0-7c09df0928a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.351001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfad60-6db1-4286-96d0-7c09df0928a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.351270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfad60-6db1-4286-96d0-7c09df0928a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.352046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfad60-6db1-4286-96d0-7c09df0928a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.353674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfad60-6db1-4286-96d0-7c09df0928a6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.364273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 789/1581] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:24.365080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfad60-6db1-4286-96d0-7c09df0928a6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.366050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 793/1582] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:24 2022] PUT /identity/v3/projects/099fe6c7868e4c2990813036624f4cd8/users/3dd192e05b7b42209a28ee1419827a6d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:24.366755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af64ec-38d1-41a9-b0ea-fc803d30bac1 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.375072 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfad60-6db1-4286-96d0-7c09df0928a6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.433095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0227c-c466-492c-b96f-00bd0745b285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.433887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0227c-c466-492c-b96f-00bd0745b285 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.434234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0227c-c466-492c-b96f-00bd0745b285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.434570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0227c-c466-492c-b96f-00bd0745b285 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.435248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0227c-c466-492c-b96f-00bd0745b285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.436462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af64ec-38d1-41a9-b0ea-fc803d30bac1 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.437240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0227c-c466-492c-b96f-00bd0745b285 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.441062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712cf34-725f-454f-9442-a42b18876048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.441661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712cf34-725f-454f-9442-a42b18876048 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.441937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712cf34-725f-454f-9442-a42b18876048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.442206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712cf34-725f-454f-9442-a42b18876048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.442834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712cf34-725f-454f-9442-a42b18876048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.444458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712cf34-725f-454f-9442-a42b18876048 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.454360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 794/1583] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:24.455254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712cf34-725f-454f-9442-a42b18876048 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.464521 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712cf34-725f-454f-9442-a42b18876048 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.469291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f7b38-f654-4076-8ef6-4d2483d82c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.469857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f7b38-f654-4076-8ef6-4d2483d82c6d admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.470159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f7b38-f654-4076-8ef6-4d2483d82c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.470968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f7b38-f654-4076-8ef6-4d2483d82c6d admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.471577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f7b38-f654-4076-8ef6-4d2483d82c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.473496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f7b38-f654-4076-8ef6-4d2483d82c6d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.559729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 790/1584] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:24 2022] POST /identity/v3/users => generated 279 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:24.560386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0227c-c466-492c-b96f-00bd0745b285 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.566777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:24.570645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0227c-c466-492c-b96f-00bd0745b285 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.575057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3075a-46c0-4ce3-a1f2-046673bb8457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.575632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3075a-46c0-4ce3-a1f2-046673bb8457 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.575905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3075a-46c0-4ce3-a1f2-046673bb8457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.576173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3075a-46c0-4ce3-a1f2-046673bb8457 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.576775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3075a-46c0-4ce3-a1f2-046673bb8457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.578718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc3075a-46c0-4ce3-a1f2-046673bb8457 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.592029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 795/1585] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:24 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:24.592842 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f7b38-f654-4076-8ef6-4d2483d82c6d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.599447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:24.603052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f7b38-f654-4076-8ef6-4d2483d82c6d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.607604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd5ba0-6f1c-492d-a32c-1f3e964a9fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.608210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd5ba0-6f1c-492d-a32c-1f3e964a9fdc admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.608484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd5ba0-6f1c-492d-a32c-1f3e964a9fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.608751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd5ba0-6f1c-492d-a32c-1f3e964a9fdc admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.609395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd5ba0-6f1c-492d-a32c-1f3e964a9fdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.611346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd5ba0-6f1c-492d-a32c-1f3e964a9fdc admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.691224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 791/1586] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:24 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:24.692775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe633f64-1a19-4462-a4a3-79b6709de1d9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.693092 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe633f64-1a19-4462-a4a3-79b6709de1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.693388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe633f64-1a19-4462-a4a3-79b6709de1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.713324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fe633f64-1a19-4462-a4a3-79b6709de1d9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:24.714722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 796/1587] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:24 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:24.714876 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fe633f64-1a19-4462-a4a3-79b6709de1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:24.719262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f723a9-30dc-4106-8641-fa6723d6fb2b None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.719692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f723a9-30dc-4106-8641-fa6723d6fb2b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.719965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f723a9-30dc-4106-8641-fa6723d6fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.723848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 792/1588] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:24.724761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe633f64-1a19-4462-a4a3-79b6709de1d9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.735101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe633f64-1a19-4462-a4a3-79b6709de1d9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.739713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eae1d7-4adc-49ae-a138-1a2b8b18a24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.741653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eae1d7-4adc-49ae-a138-1a2b8b18a24a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.741653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eae1d7-4adc-49ae-a138-1a2b8b18a24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.741653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eae1d7-4adc-49ae-a138-1a2b8b18a24a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.741653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eae1d7-4adc-49ae-a138-1a2b8b18a24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.743340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eae1d7-4adc-49ae-a138-1a2b8b18a24a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.756812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 793/1589] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:24.757027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-91f723a9-30dc-4106-8641-fa6723d6fb2b None None] MFA Rules not processed for user `3dd192e05b7b42209a28ee1419827a6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:24.757141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eae1d7-4adc-49ae-a138-1a2b8b18a24a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.757242 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-91f723a9-30dc-4106-8641-fa6723d6fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:24.767903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eae1d7-4adc-49ae-a138-1a2b8b18a24a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.773086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2777d-0675-468c-a232-e3e1a4b05075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.782026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2777d-0675-468c-a232-e3e1a4b05075 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.782026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2777d-0675-468c-a232-e3e1a4b05075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.782026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2777d-0675-468c-a232-e3e1a4b05075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.782026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2777d-0675-468c-a232-e3e1a4b05075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.782026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2777d-0675-468c-a232-e3e1a4b05075 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.787577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 794/1590] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:24.788388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2777d-0675-468c-a232-e3e1a4b05075 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.797921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2777d-0675-468c-a232-e3e1a4b05075 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.803554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335a09e-d3e8-4127-9b3c-57151f3fdff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.804150 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335a09e-d3e8-4127-9b3c-57151f3fdff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.804427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335a09e-d3e8-4127-9b3c-57151f3fdff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.804704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335a09e-d3e8-4127-9b3c-57151f3fdff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.805354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335a09e-d3e8-4127-9b3c-57151f3fdff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.806923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335a09e-d3e8-4127-9b3c-57151f3fdff4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.817652 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 795/1591] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:24.818546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335a09e-d3e8-4127-9b3c-57151f3fdff4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.823949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 797/1592] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:24.825454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f723a9-30dc-4106-8641-fa6723d6fb2b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.842922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335a09e-d3e8-4127-9b3c-57151f3fdff4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.842922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30d13d-234d-48ca-8e3e-2ca129b44de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.842922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30d13d-234d-48ca-8e3e-2ca129b44de2 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.842922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30d13d-234d-48ca-8e3e-2ca129b44de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.843314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30d13d-234d-48ca-8e3e-2ca129b44de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.843314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30d13d-234d-48ca-8e3e-2ca129b44de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.843445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f723a9-30dc-4106-8641-fa6723d6fb2b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.843535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30d13d-234d-48ca-8e3e-2ca129b44de2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.843611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89060a32-c3c5-4cde-8b26-c2f0c1a1951b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.843611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89060a32-c3c5-4cde-8b26-c2f0c1a1951b admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.843611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89060a32-c3c5-4cde-8b26-c2f0c1a1951b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.843611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89060a32-c3c5-4cde-8b26-c2f0c1a1951b admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.843899 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89060a32-c3c5-4cde-8b26-c2f0c1a1951b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.844241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89060a32-c3c5-4cde-8b26-c2f0c1a1951b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.851916 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 796/1593] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:24.852769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30d13d-234d-48ca-8e3e-2ca129b44de2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.861692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 798/1594] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:24.861692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89060a32-c3c5-4cde-8b26-c2f0c1a1951b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.867670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30d13d-234d-48ca-8e3e-2ca129b44de2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.873615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74fbfd-a20f-476c-be6f-0cca09a9505f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.874307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74fbfd-a20f-476c-be6f-0cca09a9505f admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.874658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74fbfd-a20f-476c-be6f-0cca09a9505f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.874999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74fbfd-a20f-476c-be6f-0cca09a9505f admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.875852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74fbfd-a20f-476c-be6f-0cca09a9505f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.877750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74fbfd-a20f-476c-be6f-0cca09a9505f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.889302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 797/1595] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:24.890243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74fbfd-a20f-476c-be6f-0cca09a9505f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.901597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74fbfd-a20f-476c-be6f-0cca09a9505f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.906592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727050f-4e97-407d-a742-992ecf7ecf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.907251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727050f-4e97-407d-a742-992ecf7ecf5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.907528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727050f-4e97-407d-a742-992ecf7ecf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.907800 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727050f-4e97-407d-a742-992ecf7ecf5d admin admin] PATH_INFO: `/v3/projects/4a4159dc366f41bc9e4057a079f6fb5a/users/2715cf2a5129439c83a5347bb963f1a9/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.924117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727050f-4e97-407d-a742-992ecf7ecf5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4159dc366f41bc9e4057a079f6fb5a, user_id=2715cf2a5129439c83a5347bb963f1a9, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.927775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727050f-4e97-407d-a742-992ecf7ecf5d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.931430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89060a32-c3c5-4cde-8b26-c2f0c1a1951b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.935088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d0ba7-d7d9-46c2-a079-1cc6b618d67a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.935685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d0ba7-d7d9-46c2-a079-1cc6b618d67a service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.935958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d0ba7-d7d9-46c2-a079-1cc6b618d67a service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.936223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d0ba7-d7d9-46c2-a079-1cc6b618d67a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.946093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d0ba7-d7d9-46c2-a079-1cc6b618d67a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.947533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d0ba7-d7d9-46c2-a079-1cc6b618d67a service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:24.957559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 799/1596] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:24.958324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d0ba7-d7d9-46c2-a079-1cc6b618d67a service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:24.967568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d0ba7-d7d9-46c2-a079-1cc6b618d67a service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:24.973464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78e097-48d9-44fd-88c8-9d3692e54a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:24.974116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78e097-48d9-44fd-88c8-9d3692e54a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:24.974392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78e097-48d9-44fd-88c8-9d3692e54a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:24.974663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78e097-48d9-44fd-88c8-9d3692e54a92 admin admin] PATH_INFO: `/v3/projects/901780013b03485a83261796f69e1b01/users/77cef1a9e6a241ac8401b88f4a0f69fe/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:24.989781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78e097-48d9-44fd-88c8-9d3692e54a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901780013b03485a83261796f69e1b01, user_id=77cef1a9e6a241ac8401b88f4a0f69fe, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:24.992522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78e097-48d9-44fd-88c8-9d3692e54a92 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.000213 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 798/1597] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:24 2022] PUT /identity/v3/projects/4a4159dc366f41bc9e4057a079f6fb5a/users/2715cf2a5129439c83a5347bb963f1a9/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:25.001063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727050f-4e97-407d-a742-992ecf7ecf5d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.071343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727050f-4e97-407d-a742-992ecf7ecf5d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.076047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1337-0199-4952-bd80-4b4b06467dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.076665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1337-0199-4952-bd80-4b4b06467dce admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.076976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1337-0199-4952-bd80-4b4b06467dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.077248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1337-0199-4952-bd80-4b4b06467dce admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.077883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f1337-0199-4952-bd80-4b4b06467dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.079490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f1337-0199-4952-bd80-4b4b06467dce admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.092085 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 799/1598] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:25.096516 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1337-0199-4952-bd80-4b4b06467dce admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.101233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 800/1599] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:24 2022] PUT /identity/v3/projects/901780013b03485a83261796f69e1b01/users/77cef1a9e6a241ac8401b88f4a0f69fe/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:25.102851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78e097-48d9-44fd-88c8-9d3692e54a92 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.107255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1337-0199-4952-bd80-4b4b06467dce admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.160652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:25.166719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:25.173660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad390f-dc97-452c-b0fc-d96029509f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.175286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad390f-dc97-452c-b0fc-d96029509f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.175664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad390f-dc97-452c-b0fc-d96029509f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.176252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad390f-dc97-452c-b0fc-d96029509f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.177035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad390f-dc97-452c-b0fc-d96029509f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.178720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad390f-dc97-452c-b0fc-d96029509f17 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.179018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78e097-48d9-44fd-88c8-9d3692e54a92 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.183711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732abf3-5f37-4b28-8595-8bf60c16dd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.193599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732abf3-5f37-4b28-8595-8bf60c16dd33 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.193599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732abf3-5f37-4b28-8595-8bf60c16dd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.193599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732abf3-5f37-4b28-8595-8bf60c16dd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.193599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732abf3-5f37-4b28-8595-8bf60c16dd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.193599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732abf3-5f37-4b28-8595-8bf60c16dd33 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.194063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 800/1600] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:25.194063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254106e-8813-49d5-a369-37d734b72078 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.194063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254106e-8813-49d5-a369-37d734b72078 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.194063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254106e-8813-49d5-a369-37d734b72078 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.194063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 801/1601] 10.222.0.84 () {54 vars in 962 bytes} [Wed Aug 24 11:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:31:25.195154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254106e-8813-49d5-a369-37d734b72078 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.199986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 801/1602] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:25.201289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b393a0-a4ad-4c7f-a0e8-34836c252ba1 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.201598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b393a0-a4ad-4c7f-a0e8-34836c252ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.201869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b393a0-a4ad-4c7f-a0e8-34836c252ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.209221 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254106e-8813-49d5-a369-37d734b72078 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.213921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502d330-564f-4e9f-8867-8fdea255e3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.214512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502d330-564f-4e9f-8867-8fdea255e3db admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.219472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502d330-564f-4e9f-8867-8fdea255e3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.219472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502d330-564f-4e9f-8867-8fdea255e3db admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.219472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502d330-564f-4e9f-8867-8fdea255e3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.219472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502d330-564f-4e9f-8867-8fdea255e3db admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.221480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-f1b393a0-a4ad-4c7f-a0e8-34836c252ba1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:25.222837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f1b393a0-a4ad-4c7f-a0e8-34836c252ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:25.228854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 802/1603] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:25.229632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502d330-564f-4e9f-8867-8fdea255e3db admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.233510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 802/1604] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:25.234705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c1e15-157d-4dac-bcb5-69aa656a68e5 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.234985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c1e15-157d-4dac-bcb5-69aa656a68e5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.235255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c1e15-157d-4dac-bcb5-69aa656a68e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.240092 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502d330-564f-4e9f-8867-8fdea255e3db admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.245034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41140779-6f3c-40fa-b79b-225edf7459dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.248240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41140779-6f3c-40fa-b79b-225edf7459dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.248240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41140779-6f3c-40fa-b79b-225edf7459dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.248240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41140779-6f3c-40fa-b79b-225edf7459dc admin admin] PATH_INFO: `/v3/projects/472bfc5a99a946c796dc0425f757d497/users/40b7db8901c14cbda81ce095bff44d7f/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.260890 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6b8c1e15-157d-4dac-bcb5-69aa656a68e5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:25.262087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6b8c1e15-157d-4dac-bcb5-69aa656a68e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:25.262914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41140779-6f3c-40fa-b79b-225edf7459dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=472bfc5a99a946c796dc0425f757d497, user_id=40b7db8901c14cbda81ce095bff44d7f, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.265572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41140779-6f3c-40fa-b79b-225edf7459dc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.270247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 803/1605] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:25.270861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c1e15-157d-4dac-bcb5-69aa656a68e5 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.280464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c1e15-157d-4dac-bcb5-69aa656a68e5 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.285865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0308fb5-a25b-40e1-8be1-7667ce2f406f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.286589 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0308fb5-a25b-40e1-8be1-7667ce2f406f admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.286865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0308fb5-a25b-40e1-8be1-7667ce2f406f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.287136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0308fb5-a25b-40e1-8be1-7667ce2f406f admin admin] PATH_INFO: `/v3/projects/ecec6d0c155f4d658197c4779ab9ffce/users/0031939e16194f63b99e6f27f700fa8c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.302659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0308fb5-a25b-40e1-8be1-7667ce2f406f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecec6d0c155f4d658197c4779ab9ffce, user_id=0031939e16194f63b99e6f27f700fa8c, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.305661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0308fb5-a25b-40e1-8be1-7667ce2f406f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.355558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 803/1606] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:25 2022] PUT /identity/v3/projects/472bfc5a99a946c796dc0425f757d497/users/40b7db8901c14cbda81ce095bff44d7f/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:25.357089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114a450-dd4a-4517-83dd-cb65e0e08be6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.357714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114a450-dd4a-4517-83dd-cb65e0e08be6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.357979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114a450-dd4a-4517-83dd-cb65e0e08be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.397920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0114a450-dd4a-4517-83dd-cb65e0e08be6 None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:25.399165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0114a450-dd4a-4517-83dd-cb65e0e08be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:25.407930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 804/1607] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:25 2022] PUT /identity/v3/projects/ecec6d0c155f4d658197c4779ab9ffce/users/0031939e16194f63b99e6f27f700fa8c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:25.412269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0308fb5-a25b-40e1-8be1-7667ce2f406f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.474967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0308fb5-a25b-40e1-8be1-7667ce2f406f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.475290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 804/1608] 10.222.0.84 () {58 vars in 1059 bytes} [Wed Aug 24 11:31:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:25.476106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114a450-dd4a-4517-83dd-cb65e0e08be6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.479717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4005f8-6342-4c45-a1e0-8d0614ad3b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.480516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4005f8-6342-4c45-a1e0-8d0614ad3b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.480727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4005f8-6342-4c45-a1e0-8d0614ad3b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.481031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4005f8-6342-4c45-a1e0-8d0614ad3b82 admin admin] PATH_INFO: `/v3/projects/177e00cee82c44a196eaf5d0dc07b7f2/users/4a80d0127ee349d69f0ed949bcb4c919/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.486386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114a450-dd4a-4517-83dd-cb65e0e08be6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.491104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb8a0e-1aa8-4f5c-b67a-a342539b6c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.491781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb8a0e-1aa8-4f5c-b67a-a342539b6c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.492035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb8a0e-1aa8-4f5c-b67a-a342539b6c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.492306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb8a0e-1aa8-4f5c-b67a-a342539b6c52 admin admin] PATH_INFO: `/v3/projects/1be7c9531c2f4dfca79bbeb10cf17d02/users/90c2b2941bc8460985e89cca87e7c21c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.496957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4005f8-6342-4c45-a1e0-8d0614ad3b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=177e00cee82c44a196eaf5d0dc07b7f2, user_id=4a80d0127ee349d69f0ed949bcb4c919, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.499610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4005f8-6342-4c45-a1e0-8d0614ad3b82 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.508612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb8a0e-1aa8-4f5c-b67a-a342539b6c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be7c9531c2f4dfca79bbeb10cf17d02, user_id=90c2b2941bc8460985e89cca87e7c21c, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.511348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb8a0e-1aa8-4f5c-b67a-a342539b6c52 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.576200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 805/1609] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:25 2022] PUT /identity/v3/projects/177e00cee82c44a196eaf5d0dc07b7f2/users/4a80d0127ee349d69f0ed949bcb4c919/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:25.577143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4005f8-6342-4c45-a1e0-8d0614ad3b82 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.601617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 805/1610] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:25 2022] PUT /identity/v3/projects/1be7c9531c2f4dfca79bbeb10cf17d02/users/90c2b2941bc8460985e89cca87e7c21c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:25.602387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb8a0e-1aa8-4f5c-b67a-a342539b6c52 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.603997 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-53bb8a0e-1aa8-4f5c-b67a-a342539b6c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:25.649872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4005f8-6342-4c45-a1e0-8d0614ad3b82 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.680521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb8a0e-1aa8-4f5c-b67a-a342539b6c52 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.684520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d81dff-1528-4c0a-a5b0-17b0773d4080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.685149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d81dff-1528-4c0a-a5b0-17b0773d4080 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.685424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d81dff-1528-4c0a-a5b0-17b0773d4080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.685690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d81dff-1528-4c0a-a5b0-17b0773d4080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.694881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:25.703760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b5b27-d15e-4b63-8b60-bad9d2e7333e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.704370 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b5b27-d15e-4b63-8b60-bad9d2e7333e service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.704648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b5b27-d15e-4b63-8b60-bad9d2e7333e service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.706403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b5b27-d15e-4b63-8b60-bad9d2e7333e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.714450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:25.753279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d81dff-1528-4c0a-a5b0-17b0773d4080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.754860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d81dff-1528-4c0a-a5b0-17b0773d4080 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.765551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 806/1611] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:25.766716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9976f5-0e4c-4c4f-b510-203eaf106460 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.766998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9976f5-0e4c-4c4f-b510-203eaf106460 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.767276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9976f5-0e4c-4c4f-b510-203eaf106460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.776763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b5b27-d15e-4b63-8b60-bad9d2e7333e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:25.778470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b5b27-d15e-4b63-8b60-bad9d2e7333e service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:25.790289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 806/1612] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:25.791773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1350e-dc80-49e7-9a70-f51782baafa6 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.792267 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1350e-dc80-49e7-9a70-f51782baafa6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.792635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1350e-dc80-49e7-9a70-f51782baafa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.796786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-3e9976f5-0e4c-4c4f-b510-203eaf106460 None None] MFA Rules not processed for user `40b7db8901c14cbda81ce095bff44d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:25.799912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3e9976f5-0e4c-4c4f-b510-203eaf106460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:25.815810 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9cb1350e-dc80-49e7-9a70-f51782baafa6 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:25.817233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9cb1350e-dc80-49e7-9a70-f51782baafa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:25.827319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 807/1613] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:25.828356 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1350e-dc80-49e7-9a70-f51782baafa6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:25.876058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 807/1614] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:25.880493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6aa61a-60de-4061-8376-463a7db51dee None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.880493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6aa61a-60de-4061-8376-463a7db51dee None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.880493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6aa61a-60de-4061-8376-463a7db51dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.903424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0e6aa61a-60de-4061-8376-463a7db51dee None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:25.903424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0e6aa61a-60de-4061-8376-463a7db51dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:25.916247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 808/1615] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:25.916247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3a56-75a0-4746-865b-286244a4ed07 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.916764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3a56-75a0-4746-865b-286244a4ed07 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.917273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd3a56-75a0-4746-865b-286244a4ed07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.921463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1350e-dc80-49e7-9a70-f51782baafa6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:25.929484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31895a6a-e6cc-4e76-82a1-114e6649f25c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:25.930314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31895a6a-e6cc-4e76-82a1-114e6649f25c service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:25.930718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31895a6a-e6cc-4e76-82a1-114e6649f25c service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:25.931102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31895a6a-e6cc-4e76-82a1-114e6649f25c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:25.959829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f9fd3a56-75a0-4746-865b-286244a4ed07 None None] MFA Rules not processed for user `90c2b2941bc8460985e89cca87e7c21c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:25.961257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f9fd3a56-75a0-4746-865b-286244a4ed07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:26.009391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31895a6a-e6cc-4e76-82a1-114e6649f25c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:26.011040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31895a6a-e6cc-4e76-82a1-114e6649f25c service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:26.020833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 808/1616] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:26.025992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31895a6a-e6cc-4e76-82a1-114e6649f25c service nova] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:26.038482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31895a6a-e6cc-4e76-82a1-114e6649f25c service nova] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:26.043710 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badae05-6044-4ced-8bf5-b26088ffc48b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:26.044866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badae05-6044-4ced-8bf5-b26088ffc48b service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:26.045249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badae05-6044-4ced-8bf5-b26088ffc48b service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:26.045648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badae05-6044-4ced-8bf5-b26088ffc48b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:26.075781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 809/1617] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:26.075781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3a56-75a0-4746-865b-286244a4ed07 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:26.076046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badae05-6044-4ced-8bf5-b26088ffc48b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:26.076046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badae05-6044-4ced-8bf5-b26088ffc48b service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:26.076239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd3a56-75a0-4746-865b-286244a4ed07 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:26.076239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9c959-455b-4200-b7ac-956f9735229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:26.076239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9c959-455b-4200-b7ac-956f9735229c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:26.076239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9c959-455b-4200-b7ac-956f9735229c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:26.076577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9c959-455b-4200-b7ac-956f9735229c admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:26.076577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9c959-455b-4200-b7ac-956f9735229c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:26.077886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9c959-455b-4200-b7ac-956f9735229c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:26.085694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 809/1618] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:26.087911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badae05-6044-4ced-8bf5-b26088ffc48b service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:26.108457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badae05-6044-4ced-8bf5-b26088ffc48b service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:26.113298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687843fe-0c22-44e6-a97f-b2ebcce798bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:26.114016 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687843fe-0c22-44e6-a97f-b2ebcce798bb service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:26.116442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687843fe-0c22-44e6-a97f-b2ebcce798bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:26.116442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687843fe-0c22-44e6-a97f-b2ebcce798bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:26.123398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:26.126525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687843fe-0c22-44e6-a97f-b2ebcce798bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:26.128182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687843fe-0c22-44e6-a97f-b2ebcce798bb service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:26.139956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 810/1619] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:26.139956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687843fe-0c22-44e6-a97f-b2ebcce798bb service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:26.186771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 810/1620] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:26 2022] POST /identity/v3/projects => generated 357 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:26.190834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9c959-455b-4200-b7ac-956f9735229c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:26.211647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:26.218519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687843fe-0c22-44e6-a97f-b2ebcce798bb service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:26.285927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998c1e9-34ec-4d25-9dab-3168b65243af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:26.286854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998c1e9-34ec-4d25-9dab-3168b65243af service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:26.287295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998c1e9-34ec-4d25-9dab-3168b65243af service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:26.287715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998c1e9-34ec-4d25-9dab-3168b65243af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:26.325577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9c959-455b-4200-b7ac-956f9735229c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:26.339932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d655b6a-8601-420b-8615-68488e557f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:26.340689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d655b6a-8601-420b-8615-68488e557f20 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:26.341139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d655b6a-8601-420b-8615-68488e557f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:26.341488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d655b6a-8601-420b-8615-68488e557f20 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:26.342413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d655b6a-8601-420b-8615-68488e557f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:26.344382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d655b6a-8601-420b-8615-68488e557f20 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:26.413445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998c1e9-34ec-4d25-9dab-3168b65243af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:26.415216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998c1e9-34ec-4d25-9dab-3168b65243af service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:26.430489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 811/1621] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:26.490273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 811/1622] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:26 2022] POST /identity/v3/users => generated 285 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:31:26.494614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998c1e9-34ec-4d25-9dab-3168b65243af service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:26.508507 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998c1e9-34ec-4d25-9dab-3168b65243af service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:26.515086 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0dcef-d0ca-498c-8170-b9adbc447990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:26.516562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0dcef-d0ca-498c-8170-b9adbc447990 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:26.519661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0dcef-d0ca-498c-8170-b9adbc447990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:26.520737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0dcef-d0ca-498c-8170-b9adbc447990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:26.522818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0dcef-d0ca-498c-8170-b9adbc447990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:26.527535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0dcef-d0ca-498c-8170-b9adbc447990 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:26.557191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 812/1623] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:26.562281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d655b6a-8601-420b-8615-68488e557f20 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:26.590632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d655b6a-8601-420b-8615-68488e557f20 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:26.604518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbb87f-6c26-4289-bd7b-f71ce26f18b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:26.605743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbb87f-6c26-4289-bd7b-f71ce26f18b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:26.606190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbb87f-6c26-4289-bd7b-f71ce26f18b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:26.606783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbb87f-6c26-4289-bd7b-f71ce26f18b6 admin admin] PATH_INFO: `/v3/projects/77ef209fe2d44d499ee10bdbc8000a19/users/614cf723b22d48f7a057744e34c90f25/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:26.670876 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbb87f-6c26-4289-bd7b-f71ce26f18b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ef209fe2d44d499ee10bdbc8000a19, user_id=614cf723b22d48f7a057744e34c90f25, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:26.679180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbb87f-6c26-4289-bd7b-f71ce26f18b6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:26.857038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 812/1624] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:26 2022] PUT /identity/v3/projects/77ef209fe2d44d499ee10bdbc8000a19/users/614cf723b22d48f7a057744e34c90f25/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:26.863254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0dcef-d0ca-498c-8170-b9adbc447990 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:26.914868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0dcef-d0ca-498c-8170-b9adbc447990 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:26.985278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:26.993883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf79ce-74c4-450f-b4d3-00f4457d7248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:26.994842 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf79ce-74c4-450f-b4d3-00f4457d7248 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:26.995368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf79ce-74c4-450f-b4d3-00f4457d7248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:26.995868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf79ce-74c4-450f-b4d3-00f4457d7248 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:26.996719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf79ce-74c4-450f-b4d3-00f4457d7248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:26.998627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf79ce-74c4-450f-b4d3-00f4457d7248 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:27.013166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 813/1625] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:27.019010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbb87f-6c26-4289-bd7b-f71ce26f18b6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:27.033684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbb87f-6c26-4289-bd7b-f71ce26f18b6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:27.039492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb19f75-f071-47bd-b747-0a9edfc934c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:27.040292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb19f75-f071-47bd-b747-0a9edfc934c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:27.040983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb19f75-f071-47bd-b747-0a9edfc934c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:27.041592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb19f75-f071-47bd-b747-0a9edfc934c6 admin admin] PATH_INFO: `/v3/projects/77ef209fe2d44d499ee10bdbc8000a19/users/614cf723b22d48f7a057744e34c90f25/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:27.044906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb19f75-f071-47bd-b747-0a9edfc934c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ef209fe2d44d499ee10bdbc8000a19, user_id=614cf723b22d48f7a057744e34c90f25, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:27.048236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb19f75-f071-47bd-b747-0a9edfc934c6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:27.066823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 813/1626] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:27 2022] PUT /identity/v3/projects/77ef209fe2d44d499ee10bdbc8000a19/users/614cf723b22d48f7a057744e34c90f25/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:27.087065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af2dde-61c5-4766-b7b8-7a8a9418043d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:27.091336 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af2dde-61c5-4766-b7b8-7a8a9418043d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:27.091336 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af2dde-61c5-4766-b7b8-7a8a9418043d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:27.128911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd87ed6-9f2b-4f85-b551-e5e14f3b51be None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:27.129456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd87ed6-9f2b-4f85-b551-e5e14f3b51be None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:27.129930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd87ed6-9f2b-4f85-b551-e5e14f3b51be None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:27.131620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 814/1627] 10.222.0.84 () {54 vars in 962 bytes} [Wed Aug 24 11:31:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:31:27.153469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-14af2dde-61c5-4766-b7b8-7a8a9418043d None None] MFA Rules not processed for user `614cf723b22d48f7a057744e34c90f25`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:27.155143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-14af2dde-61c5-4766-b7b8-7a8a9418043d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:27.169241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c5e94-f532-4cb5-909d-b231ffe31976 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:27.169826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c5e94-f532-4cb5-909d-b231ffe31976 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:27.170250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c5e94-f532-4cb5-909d-b231ffe31976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:27.196891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ac2c5e94-f532-4cb5-909d-b231ffe31976 None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:27.198390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ac2c5e94-f532-4cb5-909d-b231ffe31976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:27.315036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 815/1628] 10.222.0.84 () {58 vars in 1059 bytes} [Wed Aug 24 11:31:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:31:27.348441 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ee725-9e22-4405-af45-594f25648ffd None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:27.348750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ee725-9e22-4405-af45-594f25648ffd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:27.349039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ee725-9e22-4405-af45-594f25648ffd None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:27.350580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 816/1629] 10.222.0.84 () {54 vars in 912 bytes} [Wed Aug 24 11:31:27 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:31:27.351125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ee725-9e22-4405-af45-594f25648ffd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:27.352002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 814/1630] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:31:27.358827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e0bb9-2747-445b-bf55-0e5f58709331 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:27.359128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e0bb9-2747-445b-bf55-0e5f58709331 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:27.359362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e0bb9-2747-445b-bf55-0e5f58709331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:27.413754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-500e0bb9-2747-445b-bf55-0e5f58709331 None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:27.413754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-500e0bb9-2747-445b-bf55-0e5f58709331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:27.435298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 815/1631] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:31:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:27.520684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ee725-9e22-4405-af45-594f25648ffd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:27.525399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac8547-650f-40c9-8649-022aa5c57d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:27.526275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac8547-650f-40c9-8649-022aa5c57d69 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:27.526768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac8547-650f-40c9-8649-022aa5c57d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:27.527235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac8547-650f-40c9-8649-022aa5c57d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:27.541090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac8547-650f-40c9-8649-022aa5c57d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:27.543155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac8547-650f-40c9-8649-022aa5c57d69 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:27.550953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:27.559811 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 817/1632] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:27.745480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e0bb9-2747-445b-bf55-0e5f58709331 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:27.762473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e0bb9-2747-445b-bf55-0e5f58709331 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:27.769158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8071e-a606-4a31-bb3e-af09475ece5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:27.769971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8071e-a606-4a31-bb3e-af09475ece5f service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:27.770182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8071e-a606-4a31-bb3e-af09475ece5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:27.770449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8071e-a606-4a31-bb3e-af09475ece5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:27.784026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8071e-a606-4a31-bb3e-af09475ece5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:27.785814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8071e-a606-4a31-bb3e-af09475ece5f service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:27.802955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 816/1633] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:28.037448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739d624-ff53-4f47-a748-9ff7cc08986e None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:28.037972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739d624-ff53-4f47-a748-9ff7cc08986e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:28.038278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739d624-ff53-4f47-a748-9ff7cc08986e None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:28.039735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 818/1634] 10.222.0.84 () {54 vars in 912 bytes} [Wed Aug 24 11:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:31:28.045582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaad027-38b1-4b5a-a91e-d0c356ddfb29 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:28.046135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaad027-38b1-4b5a-a91e-d0c356ddfb29 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:28.046251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaad027-38b1-4b5a-a91e-d0c356ddfb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:28.284487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-4eaad027-38b1-4b5a-a91e-d0c356ddfb29 None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:28.284487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4eaad027-38b1-4b5a-a91e-d0c356ddfb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:28.284487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 817/1635] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:31:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:28.296506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739d624-ff53-4f47-a748-9ff7cc08986e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:28.296506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739d624-ff53-4f47-a748-9ff7cc08986e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:28.297310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaad027-38b1-4b5a-a91e-d0c356ddfb29 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:28.298143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed13959-6bdf-4195-940d-71a120e18a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:28.298143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed13959-6bdf-4195-940d-71a120e18a70 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:28.298143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed13959-6bdf-4195-940d-71a120e18a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:28.298143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed13959-6bdf-4195-940d-71a120e18a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:28.299796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaad027-38b1-4b5a-a91e-d0c356ddfb29 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:28.299796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ae6d1-d20a-4192-b83f-1c739cde77b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:28.299796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ae6d1-d20a-4192-b83f-1c739cde77b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:28.299796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ae6d1-d20a-4192-b83f-1c739cde77b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:28.300137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ae6d1-d20a-4192-b83f-1c739cde77b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:28.300408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed13959-6bdf-4195-940d-71a120e18a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:28.300408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed13959-6bdf-4195-940d-71a120e18a70 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:28.300408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 819/1636] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:28.378083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ae6d1-d20a-4192-b83f-1c739cde77b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:28.380183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ae6d1-d20a-4192-b83f-1c739cde77b9 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:28.403080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed13959-6bdf-4195-940d-71a120e18a70 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:28.416183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 818/1637] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:28.483106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed13959-6bdf-4195-940d-71a120e18a70 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:28.488872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:28.490155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:28.490733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:28.491383 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] PATH_INFO: `/v3/users/3dd192e05b7b42209a28ee1419827a6d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:28.493256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd192e05b7b42209a28ee1419827a6d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:28.494696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: WARNING py.warnings [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:31:28.494696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: warnings.warn(msg) Aug 24 11:31:28.494696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: Aug 24 11:31:28.496309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:28.874521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd192e05b7b42209a28ee1419827a6d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:29.034251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd192e05b7b42209a28ee1419827a6d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:29.141666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd192e05b7b42209a28ee1419827a6d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:29.142846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd192e05b7b42209a28ee1419827a6d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:29.164547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd192e05b7b42209a28ee1419827a6d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:29.172833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 820/1638] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:31:28 2022] DELETE /identity/v3/users/3dd192e05b7b42209a28ee1419827a6d => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:31:29.178429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74599b1-53bd-4cbb-885c-8f1573b906fc None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:29.181082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74599b1-53bd-4cbb-885c-8f1573b906fc None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:29.181082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74599b1-53bd-4cbb-885c-8f1573b906fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:29.226225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b74599b1-53bd-4cbb-885c-8f1573b906fc None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:29.231497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b74599b1-53bd-4cbb-885c-8f1573b906fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:29.337520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:29.362826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 819/1639] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:29.381937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:29.383744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:29.493508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d3839-6044-4a94-a619-40b13a00e9fe admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:29.498178 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f525c-1d1c-4e85-bed5-a66f267a259e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:29.499135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f525c-1d1c-4e85-bed5-a66f267a259e service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:29.499549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f525c-1d1c-4e85-bed5-a66f267a259e service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:29.500036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f525c-1d1c-4e85-bed5-a66f267a259e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:29.515380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f525c-1d1c-4e85-bed5-a66f267a259e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:29.518241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f525c-1d1c-4e85-bed5-a66f267a259e service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:29.532489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 821/1640] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:32.073517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74599b1-53bd-4cbb-885c-8f1573b906fc None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:32.075548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b74599b1-53bd-4cbb-885c-8f1573b906fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:32.100279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74599b1-53bd-4cbb-885c-8f1573b906fc None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:32.122057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:32.123137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:32.123484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:32.123753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] PATH_INFO: `/v3/projects/099fe6c7868e4c2990813036624f4cd8` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:32.127101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=099fe6c7868e4c2990813036624f4cd8)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:32.128658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: WARNING py.warnings [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 11:31:32.128658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: warnings.warn(msg) Aug 24 11:31:32.128658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: Aug 24 11:31:32.137121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:32.459949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '099fe6c7868e4c2990813036624f4cd8', 'request_id': 'req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:32.501272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '099fe6c7868e4c2990813036624f4cd8', 'request_id': 'req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:32.597665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '099fe6c7868e4c2990813036624f4cd8', 'request_id': 'req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:32.669650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] The token cache is being invalidate because project 099fe6c7868e4c2990813036624f4cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:31:32.670772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:32.674565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5159ca-f6e6-4fbd-b86d-795c8fb68d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:32.683037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 820/1641] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:31:32 2022] DELETE /identity/v3/projects/099fe6c7868e4c2990813036624f4cd8 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:32.688140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a4615-62b4-4942-a43a-bb738f53a45d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:32.688760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a4615-62b4-4942-a43a-bb738f53a45d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:32.688760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a4615-62b4-4942-a43a-bb738f53a45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:32.742439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6f0a4615-62b4-4942-a43a-bb738f53a45d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:32.743888 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6f0a4615-62b4-4942-a43a-bb738f53a45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:32.759674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:33.058657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 822/1642] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:33.061707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11130b-2b38-414a-a6ba-25c7711019d3 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:33.061987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11130b-2b38-414a-a6ba-25c7711019d3 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:33.062260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11130b-2b38-414a-a6ba-25c7711019d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:33.158462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-4d11130b-2b38-414a-a6ba-25c7711019d3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:33.158462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4d11130b-2b38-414a-a6ba-25c7711019d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:33.179492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 821/1643] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:33.183869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a4615-62b4-4942-a43a-bb738f53a45d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:33.235272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a4615-62b4-4942-a43a-bb738f53a45d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:33.264667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6e98a-1eea-44b9-8672-ea680315e7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:33.269892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6e98a-1eea-44b9-8672-ea680315e7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:33.272480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6e98a-1eea-44b9-8672-ea680315e7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:33.272480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6e98a-1eea-44b9-8672-ea680315e7d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:33.285745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6e98a-1eea-44b9-8672-ea680315e7d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:33.286090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6e98a-1eea-44b9-8672-ea680315e7d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:33.292776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6e98a-1eea-44b9-8672-ea680315e7d0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:33.380400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11130b-2b38-414a-a6ba-25c7711019d3 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:33.383708 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 823/1644] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:31:33.395668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11130b-2b38-414a-a6ba-25c7711019d3 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:33.411562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838b069-f475-4b48-a0c4-b11145315bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:33.413727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838b069-f475-4b48-a0c4-b11145315bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:33.420288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838b069-f475-4b48-a0c4-b11145315bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:33.427554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838b069-f475-4b48-a0c4-b11145315bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:33.427554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838b069-f475-4b48-a0c4-b11145315bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:33.427554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838b069-f475-4b48-a0c4-b11145315bd2 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:33.557364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 822/1645] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:33 2022] POST /identity/v3/projects => generated 363 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:33.559706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6e98a-1eea-44b9-8672-ea680315e7d0 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:33.576559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6e98a-1eea-44b9-8672-ea680315e7d0 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:33.803444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79e33-1f56-47b1-8847-f390972421cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:33.804078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79e33-1f56-47b1-8847-f390972421cb admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:33.804381 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79e33-1f56-47b1-8847-f390972421cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:33.804588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79e33-1f56-47b1-8847-f390972421cb admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:33.805260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b79e33-1f56-47b1-8847-f390972421cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:33.811642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b79e33-1f56-47b1-8847-f390972421cb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:33.948864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 824/1646] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:33 2022] POST /identity/v3/users => generated 282 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:33.951415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838b069-f475-4b48-a0c4-b11145315bd2 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:33.969319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838b069-f475-4b48-a0c4-b11145315bd2 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:33.996353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2756f-d1c7-4d5f-83fa-914c342fb5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:33.997614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2756f-d1c7-4d5f-83fa-914c342fb5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:33.997614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2756f-d1c7-4d5f-83fa-914c342fb5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:33.997614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2756f-d1c7-4d5f-83fa-914c342fb5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:33.998284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2756f-d1c7-4d5f-83fa-914c342fb5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:33.999979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2756f-d1c7-4d5f-83fa-914c342fb5f0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:34.025127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 823/1647] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:34.028958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79e33-1f56-47b1-8847-f390972421cb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:34.050644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79e33-1f56-47b1-8847-f390972421cb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:34.067535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b470ab5-3a9f-4ed0-accc-e340e8e9f6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:34.068290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b470ab5-3a9f-4ed0-accc-e340e8e9f6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:34.068633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b470ab5-3a9f-4ed0-accc-e340e8e9f6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:34.069007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b470ab5-3a9f-4ed0-accc-e340e8e9f6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:34.069854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b470ab5-3a9f-4ed0-accc-e340e8e9f6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:34.071678 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b470ab5-3a9f-4ed0-accc-e340e8e9f6a3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:34.114819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2756f-d1c7-4d5f-83fa-914c342fb5f0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:34.124079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 825/1648] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:34.132155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2756f-d1c7-4d5f-83fa-914c342fb5f0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:34.145478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d90544-7ec7-4b13-8dc5-a9727824f066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:34.147223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d90544-7ec7-4b13-8dc5-a9727824f066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:34.148155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d90544-7ec7-4b13-8dc5-a9727824f066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:34.149149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d90544-7ec7-4b13-8dc5-a9727824f066 admin admin] PATH_INFO: `/v3/projects/816d13380d7c4411a02e40118f94b5e1/users/8b60ca4bba324f9f9c9334e2ca5dc064/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:34.194515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d90544-7ec7-4b13-8dc5-a9727824f066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=816d13380d7c4411a02e40118f94b5e1, user_id=8b60ca4bba324f9f9c9334e2ca5dc064, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:34.198701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d90544-7ec7-4b13-8dc5-a9727824f066 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:34.317436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 824/1649] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:34 2022] PUT /identity/v3/projects/816d13380d7c4411a02e40118f94b5e1/users/8b60ca4bba324f9f9c9334e2ca5dc064/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:31:34.333677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c044a0-4660-4071-90ae-d72d592e5b68 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:34.334163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c044a0-4660-4071-90ae-d72d592e5b68 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:34.334589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c044a0-4660-4071-90ae-d72d592e5b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:34.397321 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-92c044a0-4660-4071-90ae-d72d592e5b68 None None] MFA Rules not processed for user `8b60ca4bba324f9f9c9334e2ca5dc064`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:34.402529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-92c044a0-4660-4071-90ae-d72d592e5b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:34.630312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 826/1650] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:34.635092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d90544-7ec7-4b13-8dc5-a9727824f066 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:34.811279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d90544-7ec7-4b13-8dc5-a9727824f066 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:34.827585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6b4dd-f704-43f2-9c4a-6117be769919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:34.827585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6b4dd-f704-43f2-9c4a-6117be769919 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:34.827585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6b4dd-f704-43f2-9c4a-6117be769919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:34.827930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6b4dd-f704-43f2-9c4a-6117be769919 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:34.828457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6b4dd-f704-43f2-9c4a-6117be769919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:34.834822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6b4dd-f704-43f2-9c4a-6117be769919 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:34.886321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:35.045500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c044a0-4660-4071-90ae-d72d592e5b68 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:35.046058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 825/1651] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:34 2022] POST /identity/v3/projects => generated 363 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:35.056822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:35.064639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c044a0-4660-4071-90ae-d72d592e5b68 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:35.252226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3997a05-b8c6-4f5b-9476-37e246cd3a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:35.254299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3997a05-b8c6-4f5b-9476-37e246cd3a3b admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:35.254976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3997a05-b8c6-4f5b-9476-37e246cd3a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:35.255399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3997a05-b8c6-4f5b-9476-37e246cd3a3b admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:35.256288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3997a05-b8c6-4f5b-9476-37e246cd3a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:35.258590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3997a05-b8c6-4f5b-9476-37e246cd3a3b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:35.419673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 827/1652] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:35 2022] POST /identity/v3/users => generated 288 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:35.426770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6b4dd-f704-43f2-9c4a-6117be769919 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:35.449109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6b4dd-f704-43f2-9c4a-6117be769919 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:35.469490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaca461-344d-4278-9f7f-1584782abae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:35.472449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaca461-344d-4278-9f7f-1584782abae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:35.473048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaca461-344d-4278-9f7f-1584782abae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:35.473544 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaca461-344d-4278-9f7f-1584782abae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:35.478015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaca461-344d-4278-9f7f-1584782abae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:35.483083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaca461-344d-4278-9f7f-1584782abae3 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:35.513293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 826/1653] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:35.514356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3997a05-b8c6-4f5b-9476-37e246cd3a3b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:35.535532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3997a05-b8c6-4f5b-9476-37e246cd3a3b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:35.552519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01535796-b00c-47a1-a8c8-6bd0059582c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:35.557466 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01535796-b00c-47a1-a8c8-6bd0059582c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:35.557741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01535796-b00c-47a1-a8c8-6bd0059582c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:35.558094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01535796-b00c-47a1-a8c8-6bd0059582c6 admin admin] PATH_INFO: `/v3/projects/9c074e34a8b24f07a25e9f8e89eda804/users/36c21f9e10a1418b843b5f1ddd785f9d/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:35.644946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01535796-b00c-47a1-a8c8-6bd0059582c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c074e34a8b24f07a25e9f8e89eda804, user_id=36c21f9e10a1418b843b5f1ddd785f9d, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:35.657076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01535796-b00c-47a1-a8c8-6bd0059582c6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:35.753921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 828/1654] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:35 2022] PUT /identity/v3/projects/9c074e34a8b24f07a25e9f8e89eda804/users/36c21f9e10a1418b843b5f1ddd785f9d/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:35.755430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaca461-344d-4278-9f7f-1584782abae3 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:35.771042 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaca461-344d-4278-9f7f-1584782abae3 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:35.941211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd5d65-8fb9-417e-a90e-a7cbb9bd4072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:35.944674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5d65-8fb9-417e-a90e-a7cbb9bd4072 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:35.944674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5d65-8fb9-417e-a90e-a7cbb9bd4072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:35.944674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5d65-8fb9-417e-a90e-a7cbb9bd4072 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:35.944674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd5d65-8fb9-417e-a90e-a7cbb9bd4072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:35.946280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd5d65-8fb9-417e-a90e-a7cbb9bd4072 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:35.978390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 827/1655] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:35.982469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01535796-b00c-47a1-a8c8-6bd0059582c6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:35.999815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01535796-b00c-47a1-a8c8-6bd0059582c6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:36.012520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1483c5-b155-4d0d-99b2-0a89f6d7eed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:36.014594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1483c5-b155-4d0d-99b2-0a89f6d7eed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:36.015211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1483c5-b155-4d0d-99b2-0a89f6d7eed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:36.020957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1483c5-b155-4d0d-99b2-0a89f6d7eed7 admin admin] PATH_INFO: `/v3/projects/9c074e34a8b24f07a25e9f8e89eda804/users/36c21f9e10a1418b843b5f1ddd785f9d/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:36.023765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1483c5-b155-4d0d-99b2-0a89f6d7eed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c074e34a8b24f07a25e9f8e89eda804, user_id=36c21f9e10a1418b843b5f1ddd785f9d, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:36.038933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1483c5-b155-4d0d-99b2-0a89f6d7eed7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:36.079122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 829/1656] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:35 2022] PUT /identity/v3/projects/9c074e34a8b24f07a25e9f8e89eda804/users/36c21f9e10a1418b843b5f1ddd785f9d/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:36.103079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f75b7-ed2f-4bee-8158-4f60f284767e None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:36.103332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f75b7-ed2f-4bee-8158-4f60f284767e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:36.103600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f75b7-ed2f-4bee-8158-4f60f284767e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:36.198145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-138f75b7-ed2f-4bee-8158-4f60f284767e None None] MFA Rules not processed for user `36c21f9e10a1418b843b5f1ddd785f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:36.200106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-138f75b7-ed2f-4bee-8158-4f60f284767e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:36.359392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 828/1657] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:36.384426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1483c5-b155-4d0d-99b2-0a89f6d7eed7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:36.387163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cc1483c5-b155-4d0d-99b2-0a89f6d7eed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:36.536438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1483c5-b155-4d0d-99b2-0a89f6d7eed7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:36.547201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5230cc-b1b5-4573-a000-30f80121dfcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:36.548066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5230cc-b1b5-4573-a000-30f80121dfcc service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:36.549980 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5230cc-b1b5-4573-a000-30f80121dfcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:36.550376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5230cc-b1b5-4573-a000-30f80121dfcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:36.572058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5230cc-b1b5-4573-a000-30f80121dfcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:36.573721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5230cc-b1b5-4573-a000-30f80121dfcc service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:36.588090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 830/1658] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:38.640641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f75b7-ed2f-4bee-8158-4f60f284767e None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:38.826802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f75b7-ed2f-4bee-8158-4f60f284767e None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:38.840732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:38.841537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:38.841826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:38.842098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] PATH_INFO: `/v3/users/8b60ca4bba324f9f9c9334e2ca5dc064` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:38.848267 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b60ca4bba324f9f9c9334e2ca5dc064)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:38.850613 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:38.858735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:39.213450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b60ca4bba324f9f9c9334e2ca5dc064'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:39.214192 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b60ca4bba324f9f9c9334e2ca5dc064'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:39.214942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b60ca4bba324f9f9c9334e2ca5dc064'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:39.237082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b60ca4bba324f9f9c9334e2ca5dc064'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:39.272858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b60ca4bba324f9f9c9334e2ca5dc064'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:39.378134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 829/1659] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:31:38 2022] DELETE /identity/v3/users/8b60ca4bba324f9f9c9334e2ca5dc064 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:39.382656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef88928-0e78-4c7a-8f9d-5dcf43913b6b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:39.383133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef88928-0e78-4c7a-8f9d-5dcf43913b6b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:39.383218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef88928-0e78-4c7a-8f9d-5dcf43913b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:39.433869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0ef88928-0e78-4c7a-8f9d-5dcf43913b6b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:39.435532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0ef88928-0e78-4c7a-8f9d-5dcf43913b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:39.622592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:39.639187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 831/1660] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:39.730435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:39.732031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:39.903127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083269f0-aae5-4467-886f-a87d8b53b803 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:39.908229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f752d7-312f-4047-84a7-09e9689c28c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:39.909298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f752d7-312f-4047-84a7-09e9689c28c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:39.909735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f752d7-312f-4047-84a7-09e9689c28c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:39.910155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f752d7-312f-4047-84a7-09e9689c28c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:39.940421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f752d7-312f-4047-84a7-09e9689c28c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:39.941900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f752d7-312f-4047-84a7-09e9689c28c1 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:39.963553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 830/1661] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:40.519184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac68c7-ee71-403e-9a11-3ce390cef85a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:40.519669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac68c7-ee71-403e-9a11-3ce390cef85a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:40.520023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac68c7-ee71-403e-9a11-3ce390cef85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:40.621790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-54ac68c7-ee71-403e-9a11-3ce390cef85a None None] MFA Rules not processed for user `2d5bfffc316f481183093e386a326946`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:40.624955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-54ac68c7-ee71-403e-9a11-3ce390cef85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:40.808417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 832/1662] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:40.820891 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefce681-f14b-4c4e-ab71-db625739b770 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:40.820891 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefce681-f14b-4c4e-ab71-db625739b770 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:40.821516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefce681-f14b-4c4e-ab71-db625739b770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:40.884224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-fefce681-f14b-4c4e-ab71-db625739b770 None None] MFA Rules not processed for user `2d5bfffc316f481183093e386a326946`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:40.885484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fefce681-f14b-4c4e-ab71-db625739b770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:40.905115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 831/1663] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:40.920379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac68c7-ee71-403e-9a11-3ce390cef85a None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:40.921900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-54ac68c7-ee71-403e-9a11-3ce390cef85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:41.129126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac68c7-ee71-403e-9a11-3ce390cef85a None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:41.139531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929b0e5-2e9c-4f72-b015-f12b1ea3e53a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:41.140448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929b0e5-2e9c-4f72-b015-f12b1ea3e53a service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:41.148395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929b0e5-2e9c-4f72-b015-f12b1ea3e53a service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:41.149502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929b0e5-2e9c-4f72-b015-f12b1ea3e53a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:41.163136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929b0e5-2e9c-4f72-b015-f12b1ea3e53a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:41.168937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929b0e5-2e9c-4f72-b015-f12b1ea3e53a service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:41.206212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 833/1664] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:42.732478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440faac-5575-45b6-8aba-da07791433e2 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:42.733502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440faac-5575-45b6-8aba-da07791433e2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:42.733995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440faac-5575-45b6-8aba-da07791433e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:42.778617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd91641-0a1b-4385-a410-67e854973e5a None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:42.779485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd91641-0a1b-4385-a410-67e854973e5a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:42.780976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd91641-0a1b-4385-a410-67e854973e5a None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:42.785275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 834/1665] 10.222.0.84 () {54 vars in 912 bytes} [Wed Aug 24 11:31:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:31:42.790216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1dd86-96ed-4d40-b977-7ac11fc59dbb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:42.790866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1dd86-96ed-4d40-b977-7ac11fc59dbb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:42.791309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1dd86-96ed-4d40-b977-7ac11fc59dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:42.808149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a440faac-5575-45b6-8aba-da07791433e2 None None] MFA Rules not processed for user `2715cf2a5129439c83a5347bb963f1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:42.809912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a440faac-5575-45b6-8aba-da07791433e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:42.881883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e4a1dd86-96ed-4d40-b977-7ac11fc59dbb None None] MFA Rules not processed for user `372e96bd3361434e9169b783f0c9ea13`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:42.883859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e4a1dd86-96ed-4d40-b977-7ac11fc59dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:42.896036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 835/1666] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:31:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:31:42.897188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1dd86-96ed-4d40-b977-7ac11fc59dbb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:42.899089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e4a1dd86-96ed-4d40-b977-7ac11fc59dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:42.924766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1dd86-96ed-4d40-b977-7ac11fc59dbb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:42.930621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:42.931704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:42.931996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:42.932269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] PATH_INFO: `/v3/projects/816d13380d7c4411a02e40118f94b5e1` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:42.941339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=816d13380d7c4411a02e40118f94b5e1)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:42.956363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:42.957789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 832/1667] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:42.964519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440faac-5575-45b6-8aba-da07791433e2 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:42.975558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a440faac-5575-45b6-8aba-da07791433e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:43.022190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440faac-5575-45b6-8aba-da07791433e2 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:43.083787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c75405-1b77-4fa3-87bb-b084b3ea5658 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:43.084417 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c75405-1b77-4fa3-87bb-b084b3ea5658 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:43.085826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c75405-1b77-4fa3-87bb-b084b3ea5658 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:43.086102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c75405-1b77-4fa3-87bb-b084b3ea5658 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:43.096733 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:43.147620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c75405-1b77-4fa3-87bb-b084b3ea5658 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:43.170178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c75405-1b77-4fa3-87bb-b084b3ea5658 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:43.204532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 833/1668] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:43.206841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c75405-1b77-4fa3-87bb-b084b3ea5658 service nova] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:43.239341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c75405-1b77-4fa3-87bb-b084b3ea5658 service nova] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:43.301566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '816d13380d7c4411a02e40118f94b5e1', 'request_id': 'req-d29172f6-f788-4bf0-a832-1908e9e67899'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:43.303061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:43.360221 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '816d13380d7c4411a02e40118f94b5e1', 'request_id': 'req-d29172f6-f788-4bf0-a832-1908e9e67899'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:43.374606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c24b3-7b86-4f45-a53a-1c5804952b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:43.375843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c24b3-7b86-4f45-a53a-1c5804952b0e service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:43.376305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c24b3-7b86-4f45-a53a-1c5804952b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:43.376581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c24b3-7b86-4f45-a53a-1c5804952b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:43.388641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c24b3-7b86-4f45-a53a-1c5804952b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:43.393021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c24b3-7b86-4f45-a53a-1c5804952b0e service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:43.417220 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 834/1669] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:43.473641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '816d13380d7c4411a02e40118f94b5e1', 'request_id': 'req-d29172f6-f788-4bf0-a832-1908e9e67899'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:43.525369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] The token cache is being invalidate because project 816d13380d7c4411a02e40118f94b5e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:31:43.525700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:43.526374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d29172f6-f788-4bf0-a832-1908e9e67899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:43.531860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 836/1670] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:31:42 2022] DELETE /identity/v3/projects/816d13380d7c4411a02e40118f94b5e1 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:43.534209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c24b3-7b86-4f45-a53a-1c5804952b0e service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:43.721992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c24b3-7b86-4f45-a53a-1c5804952b0e service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:43.729733 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:43.739822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:43.742357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:43.742357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] PATH_INFO: `/v3/users/36c21f9e10a1418b843b5f1ddd785f9d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:43.744461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36c21f9e10a1418b843b5f1ddd785f9d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:43.748112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:44.089056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36c21f9e10a1418b843b5f1ddd785f9d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:44.090021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36c21f9e10a1418b843b5f1ddd785f9d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:44.090787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36c21f9e10a1418b843b5f1ddd785f9d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:44.115082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36c21f9e10a1418b843b5f1ddd785f9d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:44.148435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36c21f9e10a1418b843b5f1ddd785f9d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:44.324234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 835/1671] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:31:43 2022] DELETE /identity/v3/users/36c21f9e10a1418b843b5f1ddd785f9d => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:44.594058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f595e87-7add-41cf-854e-8ab5e402ea2c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:44.595070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f595e87-7add-41cf-854e-8ab5e402ea2c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:44.595352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f595e87-7add-41cf-854e-8ab5e402ea2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:44.598639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:44.600244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:44.736762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7f595e87-7add-41cf-854e-8ab5e402ea2c None None] MFA Rules not processed for user `77cef1a9e6a241ac8401b88f4a0f69fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:44.738018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7f595e87-7add-41cf-854e-8ab5e402ea2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:44.837409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758e241-a342-4f2e-85fb-675e24380ae2 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:44.853700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:44.854585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:44.855032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:44.855497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] PATH_INFO: `/v3/users/90c2b2941bc8460985e89cca87e7c21c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:44.858469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c2b2941bc8460985e89cca87e7c21c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:44.862572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:44.988614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 837/1672] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:45.004136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a6c8e-2b3a-4998-8c93-e12406e11f58 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:45.004983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a6c8e-2b3a-4998-8c93-e12406e11f58 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:45.005397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a6c8e-2b3a-4998-8c93-e12406e11f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:45.071946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-501a6c8e-2b3a-4998-8c93-e12406e11f58 None None] MFA Rules not processed for user `77cef1a9e6a241ac8401b88f4a0f69fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:45.074538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:45.080973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-501a6c8e-2b3a-4998-8c93-e12406e11f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:45.106530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 838/1673] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:31:45.108473 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb79d0-2399-4cf3-ab3f-25cdb304fa49 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:45.108930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb79d0-2399-4cf3-ab3f-25cdb304fa49 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:45.109460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb79d0-2399-4cf3-ab3f-25cdb304fa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:45.207966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c2b2941bc8460985e89cca87e7c21c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:45.208649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c2b2941bc8460985e89cca87e7c21c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:45.209658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c2b2941bc8460985e89cca87e7c21c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:45.224703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c2b2941bc8460985e89cca87e7c21c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:45.244258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-34cb79d0-2399-4cf3-ab3f-25cdb304fa49 None None] MFA Rules not processed for user `0031939e16194f63b99e6f27f700fa8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:45.256214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-34cb79d0-2399-4cf3-ab3f-25cdb304fa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:45.298057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c2b2941bc8460985e89cca87e7c21c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:45.465489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 836/1674] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:31:44 2022] DELETE /identity/v3/users/90c2b2941bc8460985e89cca87e7c21c => generated 0 bytes in 864 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:45.466459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:45.473751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:45.536163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 839/1675] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:45.537935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06355493-0fd0-4df8-ab88-524e18a0c5af None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:45.538316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06355493-0fd0-4df8-ab88-524e18a0c5af None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:45.538813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06355493-0fd0-4df8-ab88-524e18a0c5af None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:45.540401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 840/1676] 10.222.0.84 () {54 vars in 961 bytes} [Wed Aug 24 11:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:31:45.549927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc37b33-d9e8-4fb4-90bc-5917a2e72e55 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:45.558086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc37b33-d9e8-4fb4-90bc-5917a2e72e55 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:45.559142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc37b33-d9e8-4fb4-90bc-5917a2e72e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:45.615949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8bc37b33-d9e8-4fb4-90bc-5917a2e72e55 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:45.617329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8bc37b33-d9e8-4fb4-90bc-5917a2e72e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:45.765783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f1631-143f-416c-aad4-5bcec87c4b4c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:45.779065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:45.792263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:45.793299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:45.793737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:45.794152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:45.797869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:45.821760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 841/1677] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:45.823304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c19cbd-ac61-42fb-bede-be6e11cb3550 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:45.825615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c19cbd-ac61-42fb-bede-be6e11cb3550 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:45.826069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c19cbd-ac61-42fb-bede-be6e11cb3550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:45.869435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-69c19cbd-ac61-42fb-bede-be6e11cb3550 None None] MFA Rules not processed for user `c7cc35f5ddf0446e90ded5c85f21f37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:45.870951 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-69c19cbd-ac61-42fb-bede-be6e11cb3550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:45.929047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:45.931456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:45.948116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 837/1678] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:46.072223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:46.073700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:46.127591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 842/1679] 10.222.0.84 () {58 vars in 1058 bytes} [Wed Aug 24 11:31:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:46.139298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adf2b8-383f-4ea7-af4b-579c88d0fbbf None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:46.145274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adf2b8-383f-4ea7-af4b-579c88d0fbbf None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:46.146282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adf2b8-383f-4ea7-af4b-579c88d0fbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:46.147237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d251a46-6820-47ad-8928-6d80897f30cb service nova] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:46.185894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41278e-0b23-433a-be7c-bcfd461e275e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:46.187343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41278e-0b23-433a-be7c-bcfd461e275e service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:46.195318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41278e-0b23-433a-be7c-bcfd461e275e service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:46.196374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41278e-0b23-433a-be7c-bcfd461e275e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:46.206900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-41adf2b8-383f-4ea7-af4b-579c88d0fbbf None None] MFA Rules not processed for user `2d5bfffc316f481183093e386a326946`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:46.209156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-41adf2b8-383f-4ea7-af4b-579c88d0fbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:46.276580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:46.350593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 843/1680] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:46.353541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adf2b8-383f-4ea7-af4b-579c88d0fbbf None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:46.359743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41278e-0b23-433a-be7c-bcfd461e275e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:46.364129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41278e-0b23-433a-be7c-bcfd461e275e service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:46.380625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adf2b8-383f-4ea7-af4b-579c88d0fbbf None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:46.386226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 838/1681] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:31:46.399394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41278e-0b23-433a-be7c-bcfd461e275e service nova] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:46.411749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b493ac-6f24-433f-b40b-29f37121d81a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:46.412039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3a41278e-0b23-433a-be7c-bcfd461e275e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:46.413320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b493ac-6f24-433f-b40b-29f37121d81a service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:46.414911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b493ac-6f24-433f-b40b-29f37121d81a service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:46.417088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b493ac-6f24-433f-b40b-29f37121d81a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:46.422076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-01b493ac-6f24-433f-b40b-29f37121d81a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:46.453697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b493ac-6f24-433f-b40b-29f37121d81a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:46.456449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b493ac-6f24-433f-b40b-29f37121d81a service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:46.511045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 844/1682] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:46.634665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41278e-0b23-433a-be7c-bcfd461e275e service nova] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:46.672248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782723e-438c-4d4c-a714-0b51d62370ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:46.672248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782723e-438c-4d4c-a714-0b51d62370ce service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:46.672248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782723e-438c-4d4c-a714-0b51d62370ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:46.672248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782723e-438c-4d4c-a714-0b51d62370ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:46.696736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782723e-438c-4d4c-a714-0b51d62370ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:46.698551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782723e-438c-4d4c-a714-0b51d62370ce service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:46.727167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 839/1683] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:47.105979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6421e9-fddf-4f5c-a3c9-9b7c146cbb91 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:47.107420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6421e9-fddf-4f5c-a3c9-9b7c146cbb91 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:47.108002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6421e9-fddf-4f5c-a3c9-9b7c146cbb91 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:47.110420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 845/1684] 10.222.0.84 () {54 vars in 912 bytes} [Wed Aug 24 11:31:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:31:47.118801 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4ef81-8925-4083-8bf3-cc9ca1bc4fb8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:47.119645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4ef81-8925-4083-8bf3-cc9ca1bc4fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:47.120365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4ef81-8925-4083-8bf3-cc9ca1bc4fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:47.163660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-72f4ef81-8925-4083-8bf3-cc9ca1bc4fb8 None None] MFA Rules not processed for user `372e96bd3361434e9169b783f0c9ea13`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:47.167113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-72f4ef81-8925-4083-8bf3-cc9ca1bc4fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:47.176988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 840/1685] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:31:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:47.180524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869013e-ff9c-48f6-a24a-0002dd96f9b3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:47.184724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869013e-ff9c-48f6-a24a-0002dd96f9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:47.184724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869013e-ff9c-48f6-a24a-0002dd96f9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:47.207430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4ef81-8925-4083-8bf3-cc9ca1bc4fb8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:47.213285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-72f4ef81-8925-4083-8bf3-cc9ca1bc4fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:47.277890 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4ef81-8925-4083-8bf3-cc9ca1bc4fb8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:47.279291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b869013e-ff9c-48f6-a24a-0002dd96f9b3 None None] MFA Rules not processed for user `4a80d0127ee349d69f0ed949bcb4c919`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:47.280704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b869013e-ff9c-48f6-a24a-0002dd96f9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:47.309995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:47.311018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:47.311688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:47.312443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] PATH_INFO: `/v3/projects/9c074e34a8b24f07a25e9f8e89eda804` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:47.314633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c074e34a8b24f07a25e9f8e89eda804)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:47.317418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:47.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 846/1686] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:47.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869013e-ff9c-48f6-a24a-0002dd96f9b3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:47.487670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b869013e-ff9c-48f6-a24a-0002dd96f9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:47.536113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869013e-ff9c-48f6-a24a-0002dd96f9b3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:47.536113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35930a65-7b2f-4a8b-8340-07700a2815af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:47.536113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35930a65-7b2f-4a8b-8340-07700a2815af service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:47.536113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35930a65-7b2f-4a8b-8340-07700a2815af service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:47.537718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35930a65-7b2f-4a8b-8340-07700a2815af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:47.546763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:47.552738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35930a65-7b2f-4a8b-8340-07700a2815af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:47.556225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35930a65-7b2f-4a8b-8340-07700a2815af service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:47.576638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 847/1687] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:47.633224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c074e34a8b24f07a25e9f8e89eda804', 'request_id': 'req-dee9d52c-378f-4387-ae26-5b1e09084596'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:47.669033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c074e34a8b24f07a25e9f8e89eda804', 'request_id': 'req-dee9d52c-378f-4387-ae26-5b1e09084596'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:47.778626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c074e34a8b24f07a25e9f8e89eda804', 'request_id': 'req-dee9d52c-378f-4387-ae26-5b1e09084596'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:47.813375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] The token cache is being invalidate because project 9c074e34a8b24f07a25e9f8e89eda804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:31:47.814583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:47.816120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-dee9d52c-378f-4387-ae26-5b1e09084596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:47.819950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 841/1688] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:31:47 2022] DELETE /identity/v3/projects/9c074e34a8b24f07a25e9f8e89eda804 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:47.826256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012ccb-4267-492e-a303-0294882e6541 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:47.826547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012ccb-4267-492e-a303-0294882e6541 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:47.826864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012ccb-4267-492e-a303-0294882e6541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:47.895866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2b012ccb-4267-492e-a303-0294882e6541 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:47.901491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2b012ccb-4267-492e-a303-0294882e6541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:48.044966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 848/1689] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:48.053853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccd1e2-6bc2-463a-96ab-1eac5469521f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:48.055424 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccd1e2-6bc2-463a-96ab-1eac5469521f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:48.056032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccd1e2-6bc2-463a-96ab-1eac5469521f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:48.099335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-bbccd1e2-6bc2-463a-96ab-1eac5469521f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:48.100480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-bbccd1e2-6bc2-463a-96ab-1eac5469521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:48.119058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 842/1690] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:31:48.123671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b012ccb-4267-492e-a303-0294882e6541 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:48.135944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b012ccb-4267-492e-a303-0294882e6541 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:48.142285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b21f1-04cc-47f5-a41c-cef0151324a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:48.143219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b21f1-04cc-47f5-a41c-cef0151324a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:48.143701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b21f1-04cc-47f5-a41c-cef0151324a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:48.144188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b21f1-04cc-47f5-a41c-cef0151324a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:48.145083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b21f1-04cc-47f5-a41c-cef0151324a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:48.145606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b21f1-04cc-47f5-a41c-cef0151324a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:48.147440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b21f1-04cc-47f5-a41c-cef0151324a5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:48.162045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 849/1691] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:31:48.166499 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccd1e2-6bc2-463a-96ab-1eac5469521f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:48.180089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccd1e2-6bc2-463a-96ab-1eac5469521f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:48.190668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3493b-6ba2-4a45-aed1-7d418e0c5ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:48.199550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3493b-6ba2-4a45-aed1-7d418e0c5ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:48.199550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3493b-6ba2-4a45-aed1-7d418e0c5ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:48.199550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3493b-6ba2-4a45-aed1-7d418e0c5ab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:48.199550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3493b-6ba2-4a45-aed1-7d418e0c5ab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:48.209107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3493b-6ba2-4a45-aed1-7d418e0c5ab6 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:48.327498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 843/1692] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:48 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:48.334370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b21f1-04cc-47f5-a41c-cef0151324a5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:48.377744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b21f1-04cc-47f5-a41c-cef0151324a5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:48.450110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:48.540267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d11bc8-9985-405f-8720-b0cfc904b138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:48.540937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d11bc8-9985-405f-8720-b0cfc904b138 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:48.541246 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d11bc8-9985-405f-8720-b0cfc904b138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:48.541546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d11bc8-9985-405f-8720-b0cfc904b138 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:48.542177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d11bc8-9985-405f-8720-b0cfc904b138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:48.544242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d11bc8-9985-405f-8720-b0cfc904b138 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:48.661146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3493b-6ba2-4a45-aed1-7d418e0c5ab6 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:48.663508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1ca3493b-6ba2-4a45-aed1-7d418e0c5ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:48.676912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 850/1693] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:48 2022] POST /identity/v3/users => generated 291 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:48.696654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d11bc8-9985-405f-8720-b0cfc904b138 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:48.725628 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d11bc8-9985-405f-8720-b0cfc904b138 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:48.730967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647a92e-c305-42f0-89eb-f2d108ac3cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:48.731699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647a92e-c305-42f0-89eb-f2d108ac3cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:48.731984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647a92e-c305-42f0-89eb-f2d108ac3cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:48.732254 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647a92e-c305-42f0-89eb-f2d108ac3cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:48.735173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647a92e-c305-42f0-89eb-f2d108ac3cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:48.735173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647a92e-c305-42f0-89eb-f2d108ac3cd8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:48.750535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 851/1694] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:48.755266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647a92e-c305-42f0-89eb-f2d108ac3cd8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:48.799398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647a92e-c305-42f0-89eb-f2d108ac3cd8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:48.813253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fbeec-c944-4db5-bef6-1f93d9a5adef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:48.815328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fbeec-c944-4db5-bef6-1f93d9a5adef admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:48.818687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fbeec-c944-4db5-bef6-1f93d9a5adef admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:48.818687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fbeec-c944-4db5-bef6-1f93d9a5adef admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:48.832601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fbeec-c944-4db5-bef6-1f93d9a5adef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:48.838169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fbeec-c944-4db5-bef6-1f93d9a5adef admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:48.894080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 852/1695] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:48.907960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fbeec-c944-4db5-bef6-1f93d9a5adef admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:48.936426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fbeec-c944-4db5-bef6-1f93d9a5adef admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:48.955456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ccdca-c4a2-4ec9-8f8b-53ea85f01bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:48.957135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ccdca-c4a2-4ec9-8f8b-53ea85f01bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:48.960353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ccdca-c4a2-4ec9-8f8b-53ea85f01bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:48.960668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ccdca-c4a2-4ec9-8f8b-53ea85f01bc9 admin admin] PATH_INFO: `/v3/projects/8cda179f486e401c9ff9df4a4ee37d70/users/2367d6e0e9bf4f52bdd6cccd455c2501/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:49.005047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3493b-6ba2-4a45-aed1-7d418e0c5ab6 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:49.033020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123fdaf-ce96-4e21-85b3-741adb0f92da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:49.033357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123fdaf-ce96-4e21-85b3-741adb0f92da service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:49.033474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123fdaf-ce96-4e21-85b3-741adb0f92da service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:49.033778 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123fdaf-ce96-4e21-85b3-741adb0f92da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:49.051359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1123fdaf-ce96-4e21-85b3-741adb0f92da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:49.073609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ccdca-c4a2-4ec9-8f8b-53ea85f01bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cda179f486e401c9ff9df4a4ee37d70, user_id=2367d6e0e9bf4f52bdd6cccd455c2501, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:49.101543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ccdca-c4a2-4ec9-8f8b-53ea85f01bc9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:49.179061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 853/1696] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:48 2022] PUT /identity/v3/projects/8cda179f486e401c9ff9df4a4ee37d70/users/2367d6e0e9bf4f52bdd6cccd455c2501/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:49.179061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ccdca-c4a2-4ec9-8f8b-53ea85f01bc9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:49.180619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a80ccdca-c4a2-4ec9-8f8b-53ea85f01bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:49.305972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:49.341532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123fdaf-ce96-4e21-85b3-741adb0f92da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:49.343275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123fdaf-ce96-4e21-85b3-741adb0f92da service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:49.394895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 844/1697] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 734 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:31:49.402063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3c9eb-dfcf-49a0-9e13-c6d44c42e1a9 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:49.405864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ccdca-c4a2-4ec9-8f8b-53ea85f01bc9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:49.406956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3c9eb-dfcf-49a0-9e13-c6d44c42e1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:49.412532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:49.413451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:49.413868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:49.414263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3c9eb-dfcf-49a0-9e13-c6d44c42e1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:49.414694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] PATH_INFO: `/v3/projects/1be7c9531c2f4dfca79bbeb10cf17d02` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:49.416049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1be7c9531c2f4dfca79bbeb10cf17d02)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:49.419432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:49.520046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9fd3c9eb-dfcf-49a0-9e13-c6d44c42e1a9 None None] MFA Rules not processed for user `2367d6e0e9bf4f52bdd6cccd455c2501`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:49.522860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9fd3c9eb-dfcf-49a0-9e13-c6d44c42e1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:49.703886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1be7c9531c2f4dfca79bbeb10cf17d02', 'request_id': 'req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:49.725654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1be7c9531c2f4dfca79bbeb10cf17d02', 'request_id': 'req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:49.842804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 845/1698] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:49.847379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3c9eb-dfcf-49a0-9e13-c6d44c42e1a9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:49.882512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3c9eb-dfcf-49a0-9e13-c6d44c42e1a9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:49.936924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1be7c9531c2f4dfca79bbeb10cf17d02', 'request_id': 'req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:49.972525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] The token cache is being invalidate because project 1be7c9531c2f4dfca79bbeb10cf17d02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:31:49.973646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:49.975850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8cbff9d-6e3f-41aa-8ce6-a00af4c3e74c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:49.995122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 854/1699] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:31:49 2022] DELETE /identity/v3/projects/1be7c9531c2f4dfca79bbeb10cf17d02 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:50.005249 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748339f-fb8f-42b1-8367-76b093a73a33 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:50.005583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748339f-fb8f-42b1-8367-76b093a73a33 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:50.005872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748339f-fb8f-42b1-8367-76b093a73a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:50.099606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d748339f-fb8f-42b1-8367-76b093a73a33 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:50.101699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d748339f-fb8f-42b1-8367-76b093a73a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:50.133044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b2538-4f6b-4afa-8f84-947b3ad05747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:50.134873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b2538-4f6b-4afa-8f84-947b3ad05747 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:50.140120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b2538-4f6b-4afa-8f84-947b3ad05747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:50.140441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b2538-4f6b-4afa-8f84-947b3ad05747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:50.141321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b2538-4f6b-4afa-8f84-947b3ad05747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:50.144148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b2538-4f6b-4afa-8f84-947b3ad05747 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:50.317852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 846/1700] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:49 2022] POST /identity/v3/projects => generated 385 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:31:50.322368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b2538-4f6b-4afa-8f84-947b3ad05747 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:50.328831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-771b2538-4f6b-4afa-8f84-947b3ad05747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:50.398360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 855/1701] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:50.412882 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a97f1-a448-4330-b7bf-5f90eb8c91cb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:50.413365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a97f1-a448-4330-b7bf-5f90eb8c91cb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:50.413849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a97f1-a448-4330-b7bf-5f90eb8c91cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:50.463777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6f8a97f1-a448-4330-b7bf-5f90eb8c91cb None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:50.465168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6f8a97f1-a448-4330-b7bf-5f90eb8c91cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:50.549581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b2538-4f6b-4afa-8f84-947b3ad05747 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:50.558185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1f21f-92ab-4b80-a812-b3829116e489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:50.560370 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1f21f-92ab-4b80-a812-b3829116e489 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:50.561613 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1f21f-92ab-4b80-a812-b3829116e489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:50.562120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1f21f-92ab-4b80-a812-b3829116e489 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:50.564741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1f21f-92ab-4b80-a812-b3829116e489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:50.570103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1f21f-92ab-4b80-a812-b3829116e489 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:50.672369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 856/1702] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:31:50.676944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a97f1-a448-4330-b7bf-5f90eb8c91cb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:50.695917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a97f1-a448-4330-b7bf-5f90eb8c91cb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:50.711424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330089e-cfbf-439d-9f15-077bed5ab18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:50.712777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330089e-cfbf-439d-9f15-077bed5ab18c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:50.714066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330089e-cfbf-439d-9f15-077bed5ab18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:50.717292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330089e-cfbf-439d-9f15-077bed5ab18c admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:50.718220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330089e-cfbf-439d-9f15-077bed5ab18c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:31:50.718709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330089e-cfbf-439d-9f15-077bed5ab18c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:50.720633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330089e-cfbf-439d-9f15-077bed5ab18c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:50.746744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 857/1703] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:31:50.752030 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330089e-cfbf-439d-9f15-077bed5ab18c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:50.778923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 847/1704] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:50 2022] POST /identity/v3/users => generated 299 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:50.783734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330089e-cfbf-439d-9f15-077bed5ab18c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:50.788901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1f21f-92ab-4b80-a812-b3829116e489 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:50.803615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568836f5-f988-452a-aa16-26215a1e6aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:50.804307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568836f5-f988-452a-aa16-26215a1e6aeb admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:50.804587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568836f5-f988-452a-aa16-26215a1e6aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:50.806025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1f21f-92ab-4b80-a812-b3829116e489 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:50.806774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568836f5-f988-452a-aa16-26215a1e6aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:50.807758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568836f5-f988-452a-aa16-26215a1e6aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:50.812758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568836f5-f988-452a-aa16-26215a1e6aeb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:50.820311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebd592-4aed-4360-adcc-25b4b2fd8ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:50.821349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebd592-4aed-4360-adcc-25b4b2fd8ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:50.822388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebd592-4aed-4360-adcc-25b4b2fd8ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:50.822679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebd592-4aed-4360-adcc-25b4b2fd8ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:50.823338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebd592-4aed-4360-adcc-25b4b2fd8ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:50.825095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebd592-4aed-4360-adcc-25b4b2fd8ae6 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:50.849035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 848/1705] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:50.852070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebd592-4aed-4360-adcc-25b4b2fd8ae6 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:50.870427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebd592-4aed-4360-adcc-25b4b2fd8ae6 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:50.879129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44aa05-70b4-4849-8084-8e4566811872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:50.880015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44aa05-70b4-4849-8084-8e4566811872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:50.880403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44aa05-70b4-4849-8084-8e4566811872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:50.882690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44aa05-70b4-4849-8084-8e4566811872 admin admin] PATH_INFO: `/v3/projects/3d3a4c4dd27c45b19aed9ab8be3cea93/users/1d7f5dbaa48841e9a59fc53e3a2a6a94/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:50.944011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44aa05-70b4-4849-8084-8e4566811872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3a4c4dd27c45b19aed9ab8be3cea93, user_id=1d7f5dbaa48841e9a59fc53e3a2a6a94, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:50.946877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44aa05-70b4-4849-8084-8e4566811872 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:50.956002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 858/1706] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:50 2022] POST /identity/v3/projects => generated 403 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:50.958929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568836f5-f988-452a-aa16-26215a1e6aeb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:50.965276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:51.077264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 849/1707] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:50 2022] PUT /identity/v3/projects/3d3a4c4dd27c45b19aed9ab8be3cea93/users/1d7f5dbaa48841e9a59fc53e3a2a6a94/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:31:51.087314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44aa05-70b4-4849-8084-8e4566811872 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:51.108162 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568836f5-f988-452a-aa16-26215a1e6aeb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:51.215258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44aa05-70b4-4849-8084-8e4566811872 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:51.224126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6a2f5-5fe7-4883-b384-23cca9045c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:51.225985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6a2f5-5fe7-4883-b384-23cca9045c23 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:51.226628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6a2f5-5fe7-4883-b384-23cca9045c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:51.227562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6a2f5-5fe7-4883-b384-23cca9045c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:51.228787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6a2f5-5fe7-4883-b384-23cca9045c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:51.231217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6a2f5-5fe7-4883-b384-23cca9045c23 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:51.233300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d6b9b-b4b6-4f3e-a53a-4217d49c870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:51.233995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d6b9b-b4b6-4f3e-a53a-4217d49c870a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:51.234265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d6b9b-b4b6-4f3e-a53a-4217d49c870a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:51.234704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d6b9b-b4b6-4f3e-a53a-4217d49c870a admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:51.235684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d6b9b-b4b6-4f3e-a53a-4217d49c870a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:51.238867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d6b9b-b4b6-4f3e-a53a-4217d49c870a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:51.246101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 850/1708] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:31:51.252623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6a2f5-5fe7-4883-b384-23cca9045c23 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:51.280300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6a2f5-5fe7-4883-b384-23cca9045c23 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:51.297391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b953da-ae41-4ae4-88b2-c0a40c334cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:51.298789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b953da-ae41-4ae4-88b2-c0a40c334cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:51.299242 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b953da-ae41-4ae4-88b2-c0a40c334cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:51.301462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b953da-ae41-4ae4-88b2-c0a40c334cb2 admin admin] PATH_INFO: `/v3/projects/3d3a4c4dd27c45b19aed9ab8be3cea93/users/1d7f5dbaa48841e9a59fc53e3a2a6a94/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:51.305702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b953da-ae41-4ae4-88b2-c0a40c334cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3a4c4dd27c45b19aed9ab8be3cea93, user_id=1d7f5dbaa48841e9a59fc53e3a2a6a94, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:51.312357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b953da-ae41-4ae4-88b2-c0a40c334cb2 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:51.338592 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 851/1709] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:51 2022] PUT /identity/v3/projects/3d3a4c4dd27c45b19aed9ab8be3cea93/users/1d7f5dbaa48841e9a59fc53e3a2a6a94/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:51.360836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef609-92f2-4df2-b93b-0f0f023e3330 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:51.360836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef609-92f2-4df2-b93b-0f0f023e3330 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:51.360836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef609-92f2-4df2-b93b-0f0f023e3330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:51.371414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:51.383024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 859/1710] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:50 2022] POST /identity/v3/users => generated 302 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:51.388033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d6b9b-b4b6-4f3e-a53a-4217d49c870a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:51.411686 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d6b9b-b4b6-4f3e-a53a-4217d49c870a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:51.425339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-f2bef609-92f2-4df2-b93b-0f0f023e3330 None None] MFA Rules not processed for user `1d7f5dbaa48841e9a59fc53e3a2a6a94`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:51.426739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f2bef609-92f2-4df2-b93b-0f0f023e3330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:51.526337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822d4a2-15d5-4cb3-9ef2-48c571a9fa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:51.526962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822d4a2-15d5-4cb3-9ef2-48c571a9fa56 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:51.527237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822d4a2-15d5-4cb3-9ef2-48c571a9fa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:51.527523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822d4a2-15d5-4cb3-9ef2-48c571a9fa56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:51.528294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822d4a2-15d5-4cb3-9ef2-48c571a9fa56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:51.530054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822d4a2-15d5-4cb3-9ef2-48c571a9fa56 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:51.537229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 852/1711] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:31:51.559665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef609-92f2-4df2-b93b-0f0f023e3330 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:51.562811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f2bef609-92f2-4df2-b93b-0f0f023e3330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:51.571392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 860/1712] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:51.572566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822d4a2-15d5-4cb3-9ef2-48c571a9fa56 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:51.618661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822d4a2-15d5-4cb3-9ef2-48c571a9fa56 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:51.636729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781aaea-4b66-45b7-911e-a4500cf6dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:51.637389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781aaea-4b66-45b7-911e-a4500cf6dae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:51.653862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781aaea-4b66-45b7-911e-a4500cf6dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:51.654069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781aaea-4b66-45b7-911e-a4500cf6dae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:51.655559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781aaea-4b66-45b7-911e-a4500cf6dae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:51.660417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781aaea-4b66-45b7-911e-a4500cf6dae1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:51.685628 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 861/1713] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:51.686802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781aaea-4b66-45b7-911e-a4500cf6dae1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:51.688432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef609-92f2-4df2-b93b-0f0f023e3330 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:51.698945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25566694-6cf2-42f2-aa1f-5e31973fed57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:51.700881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25566694-6cf2-42f2-aa1f-5e31973fed57 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:51.701514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25566694-6cf2-42f2-aa1f-5e31973fed57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:51.701797 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25566694-6cf2-42f2-aa1f-5e31973fed57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:51.705644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781aaea-4b66-45b7-911e-a4500cf6dae1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:51.715610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32c320-c7f1-4137-9c90-40c02ade4579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:51.716554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32c320-c7f1-4137-9c90-40c02ade4579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:51.717123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32c320-c7f1-4137-9c90-40c02ade4579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:51.717599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32c320-c7f1-4137-9c90-40c02ade4579 admin admin] PATH_INFO: `/v3/projects/053e970b30924e9da928008887ec8273/users/e156692963da4610be615703daaef24d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:51.734241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25566694-6cf2-42f2-aa1f-5e31973fed57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:51.735976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25566694-6cf2-42f2-aa1f-5e31973fed57 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:51.744753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32c320-c7f1-4137-9c90-40c02ade4579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053e970b30924e9da928008887ec8273, user_id=e156692963da4610be615703daaef24d, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:51.748025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32c320-c7f1-4137-9c90-40c02ade4579 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:51.767504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 853/1714] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:51.856168 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd3a93-6e80-40d2-9c3f-e952bb2420f1 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:51.856825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd3a93-6e80-40d2-9c3f-e952bb2420f1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:51.857425 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd3a93-6e80-40d2-9c3f-e952bb2420f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:51.889881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ecbd3a93-6e80-40d2-9c3f-e952bb2420f1 None None] MFA Rules not processed for user `2715cf2a5129439c83a5347bb963f1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:51.891286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ecbd3a93-6e80-40d2-9c3f-e952bb2420f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:51.924825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 862/1715] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:51 2022] PUT /identity/v3/projects/053e970b30924e9da928008887ec8273/users/e156692963da4610be615703daaef24d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:51.926721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32c320-c7f1-4137-9c90-40c02ade4579 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:52.008812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 854/1716] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:52.010388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ff121-43b8-4862-9f76-a801e959a6f6 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:52.010808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ff121-43b8-4862-9f76-a801e959a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:52.011228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ff121-43b8-4862-9f76-a801e959a6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:52.082740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-994ff121-43b8-4862-9f76-a801e959a6f6 None None] MFA Rules not processed for user `e156692963da4610be615703daaef24d`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:52.084217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-994ff121-43b8-4862-9f76-a801e959a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:52.176418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32c320-c7f1-4137-9c90-40c02ade4579 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:52.184579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:52.185278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:52.185709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:52.185993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] PATH_INFO: `/v3/users/2367d6e0e9bf4f52bdd6cccd455c2501` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:52.187732 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2367d6e0e9bf4f52bdd6cccd455c2501)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:52.190435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:52.313685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 855/1717] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:52.332538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ff121-43b8-4862-9f76-a801e959a6f6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:52.375681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ff121-43b8-4862-9f76-a801e959a6f6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:52.392584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568eea6-2419-4199-8c11-3466eefc9948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:52.397028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568eea6-2419-4199-8c11-3466eefc9948 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:52.397428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568eea6-2419-4199-8c11-3466eefc9948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:52.397777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568eea6-2419-4199-8c11-3466eefc9948 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:52.398970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568eea6-2419-4199-8c11-3466eefc9948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:52.401093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568eea6-2419-4199-8c11-3466eefc9948 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:52.468364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2367d6e0e9bf4f52bdd6cccd455c2501'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:52.551023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 856/1718] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:31:52 2022] POST /identity/v3/projects => generated 403 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:31:52.552004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568eea6-2419-4199-8c11-3466eefc9948 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:52.566157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2367d6e0e9bf4f52bdd6cccd455c2501'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:52.593553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2367d6e0e9bf4f52bdd6cccd455c2501'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:52.599338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2367d6e0e9bf4f52bdd6cccd455c2501'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:52.615946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f32d4ee1-da9e-47b4-b68c-0d2a6b817f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2367d6e0e9bf4f52bdd6cccd455c2501'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:52.631067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 863/1719] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:31:51 2022] DELETE /identity/v3/users/2367d6e0e9bf4f52bdd6cccd455c2501 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:52.634920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2cf82-909d-4ccc-ba45-8726c7589cde None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:52.634920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2cf82-909d-4ccc-ba45-8726c7589cde None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:52.634920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2cf82-909d-4ccc-ba45-8726c7589cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:52.676036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-78e2cf82-909d-4ccc-ba45-8726c7589cde None None] MFA Rules not processed for user `0031939e16194f63b99e6f27f700fa8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:52.677400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-78e2cf82-909d-4ccc-ba45-8726c7589cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:52.805326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 864/1720] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:52.806483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d3041-858a-46ee-b386-ef3fd01afcd1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:52.806825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d3041-858a-46ee-b386-ef3fd01afcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:52.807629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d3041-858a-46ee-b386-ef3fd01afcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:52.815525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568eea6-2419-4199-8c11-3466eefc9948 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:52.897953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-742d3041-858a-46ee-b386-ef3fd01afcd1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:52.899197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-742d3041-858a-46ee-b386-ef3fd01afcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:52.946155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc27b3-b81f-4d88-a351-0fa99a7ef80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:52.946535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc27b3-b81f-4d88-a351-0fa99a7ef80b admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:52.946821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc27b3-b81f-4d88-a351-0fa99a7ef80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:52.947093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc27b3-b81f-4d88-a351-0fa99a7ef80b admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:52.947728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc27b3-b81f-4d88-a351-0fa99a7ef80b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:52.949717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc27b3-b81f-4d88-a351-0fa99a7ef80b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:53.014011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 865/1721] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:53.015742 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b983d-d529-43c4-abd3-78eb5db2a49f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:53.016032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b983d-d529-43c4-abd3-78eb5db2a49f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:53.016309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b983d-d529-43c4-abd3-78eb5db2a49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:53.108406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c24b983d-d529-43c4-abd3-78eb5db2a49f None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:53.110466 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c24b983d-d529-43c4-abd3-78eb5db2a49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:53.172302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 857/1722] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:31:52 2022] POST /identity/v3/users => generated 308 bytes in 621 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:31:53.174329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc27b3-b81f-4d88-a351-0fa99a7ef80b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:53.181854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fbdc27b3-b81f-4d88-a351-0fa99a7ef80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:53.268566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:53.319721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 866/1723] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:31:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:53.324567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b983d-d529-43c4-abd3-78eb5db2a49f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:53.325478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c24b983d-d529-43c4-abd3-78eb5db2a49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:53.360718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc27b3-b81f-4d88-a351-0fa99a7ef80b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:53.369940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc2983-47f7-4da0-9281-6f1a233b8395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:53.370846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2983-47f7-4da0-9281-6f1a233b8395 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:53.370977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b983d-d529-43c4-abd3-78eb5db2a49f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:53.371280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2983-47f7-4da0-9281-6f1a233b8395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:53.371750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2983-47f7-4da0-9281-6f1a233b8395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:53.389082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc2983-47f7-4da0-9281-6f1a233b8395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:53.391464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc2983-47f7-4da0-9281-6f1a233b8395 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:53.404012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 858/1724] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:53.408504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1a0e4-d9ca-43ca-8b7f-b692c78a47ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:53.409697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1a0e4-d9ca-43ca-8b7f-b692c78a47ba admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:53.410202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1a0e4-d9ca-43ca-8b7f-b692c78a47ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:53.410599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1a0e4-d9ca-43ca-8b7f-b692c78a47ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:53.411294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1a0e4-d9ca-43ca-8b7f-b692c78a47ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:53.413021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1a0e4-d9ca-43ca-8b7f-b692c78a47ba admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:53.440410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc2983-47f7-4da0-9281-6f1a233b8395 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:53.451451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 867/1725] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:53.457246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc2983-47f7-4da0-9281-6f1a233b8395 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:53.466778 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caee192-f923-4ad4-b059-d4bf2b0e662a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:53.468064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee192-f923-4ad4-b059-d4bf2b0e662a admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:53.470757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee192-f923-4ad4-b059-d4bf2b0e662a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:53.470757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee192-f923-4ad4-b059-d4bf2b0e662a admin admin] PATH_INFO: `/v3/projects/ccdea3274248427f947b59b76f1a54f8/users/b2c78dea998f471296b100b8bfdc1d6b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:53.498299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caee192-f923-4ad4-b059-d4bf2b0e662a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccdea3274248427f947b59b76f1a54f8, user_id=b2c78dea998f471296b100b8bfdc1d6b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:53.503217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caee192-f923-4ad4-b059-d4bf2b0e662a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:53.681932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 859/1726] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:53 2022] PUT /identity/v3/projects/ccdea3274248427f947b59b76f1a54f8/users/b2c78dea998f471296b100b8bfdc1d6b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:53.694468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1a0e4-d9ca-43ca-8b7f-b692c78a47ba admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:53.720652 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1a0e4-d9ca-43ca-8b7f-b692c78a47ba admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:53.840380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f4778-30ca-485b-8663-14038a2bff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:53.841022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f4778-30ca-485b-8663-14038a2bff6a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:53.841855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f4778-30ca-485b-8663-14038a2bff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:53.841855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f4778-30ca-485b-8663-14038a2bff6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:53.842327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f4778-30ca-485b-8663-14038a2bff6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:53.843850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f4778-30ca-485b-8663-14038a2bff6a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:53.856145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 868/1727] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:31:53.860574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caee192-f923-4ad4-b059-d4bf2b0e662a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:53.876139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caee192-f923-4ad4-b059-d4bf2b0e662a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:53.881134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a9d35-edd6-49fe-8188-63a0e268ad0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:53.882574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a9d35-edd6-49fe-8188-63a0e268ad0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:53.882574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a9d35-edd6-49fe-8188-63a0e268ad0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:53.882574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a9d35-edd6-49fe-8188-63a0e268ad0d admin admin] PATH_INFO: `/v3/projects/ccdea3274248427f947b59b76f1a54f8/users/b2c78dea998f471296b100b8bfdc1d6b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:53.884655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a9d35-edd6-49fe-8188-63a0e268ad0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccdea3274248427f947b59b76f1a54f8, user_id=b2c78dea998f471296b100b8bfdc1d6b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:53.887546 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a9d35-edd6-49fe-8188-63a0e268ad0d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:53.908943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 860/1728] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:31:53 2022] PUT /identity/v3/projects/ccdea3274248427f947b59b76f1a54f8/users/b2c78dea998f471296b100b8bfdc1d6b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:53.911493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7dc64-b6d2-431d-9bf9-b2ea44ab97a8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:53.911782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7dc64-b6d2-431d-9bf9-b2ea44ab97a8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:53.912053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7dc64-b6d2-431d-9bf9-b2ea44ab97a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:53.972613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-adb7dc64-b6d2-431d-9bf9-b2ea44ab97a8 None None] MFA Rules not processed for user `b2c78dea998f471296b100b8bfdc1d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:53.983098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-adb7dc64-b6d2-431d-9bf9-b2ea44ab97a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:53.983797 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a9d35-edd6-49fe-8188-63a0e268ad0d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:53.986029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9f0a9d35-edd6-49fe-8188-63a0e268ad0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:54.206139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 869/1729] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:53 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:54.215060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7dc64-b6d2-431d-9bf9-b2ea44ab97a8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:54.227608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-adb7dc64-b6d2-431d-9bf9-b2ea44ab97a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:54.361656 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a9d35-edd6-49fe-8188-63a0e268ad0d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:54.389912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930e54c-be3b-4049-b4c5-1913452f6a43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:54.390735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930e54c-be3b-4049-b4c5-1913452f6a43 service swift] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:54.391186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930e54c-be3b-4049-b4c5-1913452f6a43 service swift] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:54.392204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930e54c-be3b-4049-b4c5-1913452f6a43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:54.418629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:54.529662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930e54c-be3b-4049-b4c5-1913452f6a43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:54.531293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930e54c-be3b-4049-b4c5-1913452f6a43 service swift] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:54.541504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7dc64-b6d2-431d-9bf9-b2ea44ab97a8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:54.550955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad1e82-77b3-4414-a8c0-113aa0e6a133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:54.552036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad1e82-77b3-4414-a8c0-113aa0e6a133 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:54.552463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad1e82-77b3-4414-a8c0-113aa0e6a133 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:54.552903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad1e82-77b3-4414-a8c0-113aa0e6a133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:54.562622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 861/1730] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:31:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 11:31:54.591371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad1e82-77b3-4414-a8c0-113aa0e6a133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:54.594319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad1e82-77b3-4414-a8c0-113aa0e6a133 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:54.615565 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 870/1731] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:54 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:54.993880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930e54c-be3b-4049-b4c5-1913452f6a43 service swift] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:55.088964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:55.205834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930e54c-be3b-4049-b4c5-1913452f6a43 service swift] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:55.227636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:55.228323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:55.228606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:55.228969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] PATH_INFO: `/v3/users/e156692963da4610be615703daaef24d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:55.234973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e156692963da4610be615703daaef24d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:55.237042 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:55.805397 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e156692963da4610be615703daaef24d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:55.806232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e156692963da4610be615703daaef24d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:55.809356 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e156692963da4610be615703daaef24d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:55.881762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e156692963da4610be615703daaef24d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:55.960865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90720ae6-b2d1-476d-8525-a3c59b60fb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e156692963da4610be615703daaef24d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:56.073555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 862/1732] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:31:54 2022] DELETE /identity/v3/users/e156692963da4610be615703daaef24d => generated 0 bytes in 1077 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:56.075301 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc53e-a923-46db-ba4c-da38c993e724 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:56.075742 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc53e-a923-46db-ba4c-da38c993e724 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:56.076033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc53e-a923-46db-ba4c-da38c993e724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:56.082566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:56.147626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-15abc53e-a923-46db-ba4c-da38c993e724 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:56.169361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-15abc53e-a923-46db-ba4c-da38c993e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:56.403894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 871/1733] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:56.435056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a97f04-5f82-4b72-9adb-94fa84186fb6 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:56.435766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a97f04-5f82-4b72-9adb-94fa84186fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:56.436317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a97f04-5f82-4b72-9adb-94fa84186fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:56.470718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abc53e-a923-46db-ba4c-da38c993e724 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:56.472528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-15abc53e-a923-46db-ba4c-da38c993e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:56.525867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-74a97f04-5f82-4b72-9adb-94fa84186fb6 None None] MFA Rules not processed for user `372e96bd3361434e9169b783f0c9ea13`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:56.529045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-74a97f04-5f82-4b72-9adb-94fa84186fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:56.637756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abc53e-a923-46db-ba4c-da38c993e724 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:56.645739 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480feff7-33d3-4ce8-ae75-c6754a9021d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:56.651857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480feff7-33d3-4ce8-ae75-c6754a9021d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:56.652163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480feff7-33d3-4ce8-ae75-c6754a9021d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:56.652620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480feff7-33d3-4ce8-ae75-c6754a9021d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:56.655593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 863/1734] 10.222.0.94 () {58 vars in 1015 bytes} [Wed Aug 24 11:31:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:56.656739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a97f04-5f82-4b72-9adb-94fa84186fb6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:56.661916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-74a97f04-5f82-4b72-9adb-94fa84186fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:56.692266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a97f04-5f82-4b72-9adb-94fa84186fb6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:56.697888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:56.699102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:56.699864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:56.700522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] PATH_INFO: `/v3/projects/8cda179f486e401c9ff9df4a4ee37d70` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:56.709869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cda179f486e401c9ff9df4a4ee37d70)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:56.711717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:56.715422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480feff7-33d3-4ce8-ae75-c6754a9021d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:56.732391 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480feff7-33d3-4ce8-ae75-c6754a9021d6 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:56.758892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 872/1735] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:56.761004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480feff7-33d3-4ce8-ae75-c6754a9021d6 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:56.778099 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480feff7-33d3-4ce8-ae75-c6754a9021d6 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:56.783121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36995d-951f-41d8-ae80-42a36feda931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:56.783912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36995d-951f-41d8-ae80-42a36feda931 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:56.784351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36995d-951f-41d8-ae80-42a36feda931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:56.784634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36995d-951f-41d8-ae80-42a36feda931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:56.811852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36995d-951f-41d8-ae80-42a36feda931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:56.814225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36995d-951f-41d8-ae80-42a36feda931 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:56.830602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 873/1736] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:31:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:31:57.037889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:57.091770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cda179f486e401c9ff9df4a4ee37d70', 'request_id': 'req-d0481d77-06ef-402e-970b-f05363dcbef1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:57.098663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cda179f486e401c9ff9df4a4ee37d70', 'request_id': 'req-d0481d77-06ef-402e-970b-f05363dcbef1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:57.228721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cda179f486e401c9ff9df4a4ee37d70', 'request_id': 'req-d0481d77-06ef-402e-970b-f05363dcbef1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:57.258291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] The token cache is being invalidate because project 8cda179f486e401c9ff9df4a4ee37d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:31:57.258818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:57.260134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d0481d77-06ef-402e-970b-f05363dcbef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:57.265385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 864/1737] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:31:56 2022] DELETE /identity/v3/projects/8cda179f486e401c9ff9df4a4ee37d70 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:57.267021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36995d-951f-41d8-ae80-42a36feda931 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:57.289872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36995d-951f-41d8-ae80-42a36feda931 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:57.472610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:57.477489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:57.477489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:57.477489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] PATH_INFO: `/v3/users/1d7f5dbaa48841e9a59fc53e3a2a6a94` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:57.477489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d7f5dbaa48841e9a59fc53e3a2a6a94)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:57.477489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:58.031674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d7f5dbaa48841e9a59fc53e3a2a6a94'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:58.254561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d7f5dbaa48841e9a59fc53e3a2a6a94'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:58.307345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d7f5dbaa48841e9a59fc53e3a2a6a94'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:58.308791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d7f5dbaa48841e9a59fc53e3a2a6a94'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:58.339215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d7f5dbaa48841e9a59fc53e3a2a6a94'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:31:58.355251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 874/1738] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:31:57 2022] DELETE /identity/v3/users/1d7f5dbaa48841e9a59fc53e3a2a6a94 => generated 0 bytes in 1089 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:31:59.028560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a5cf0-f727-46c0-96f3-37f5b4a3f9f7 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:59.029173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a5cf0-f727-46c0-96f3-37f5b4a3f9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:59.029729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a5cf0-f727-46c0-96f3-37f5b4a3f9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:59.078711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a64a5cf0-f727-46c0-96f3-37f5b4a3f9f7 None None] MFA Rules not processed for user `4a80d0127ee349d69f0ed949bcb4c919`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:31:59.081676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a64a5cf0-f727-46c0-96f3-37f5b4a3f9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:59.117208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:59.268345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 865/1739] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:31:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:31:59.428470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:31:59.596442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:59.600684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:59.683608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a5cf0-f727-46c0-96f3-37f5b4a3f9f7 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:31:59.685685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a64a5cf0-f727-46c0-96f3-37f5b4a3f9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:31:59.813957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4274fa-d6c8-4699-8f90-6cc54ef62802 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:59.827347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:59.828581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:59.829757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:59.830177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] PATH_INFO: `/v3/projects/053e970b30924e9da928008887ec8273` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:59.836507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=053e970b30924e9da928008887ec8273)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:59.839094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:31:59.860183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a5cf0-f727-46c0-96f3-37f5b4a3f9f7 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:31:59.873691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:31:59.874359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:31:59.874637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:31:59.874908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] PATH_INFO: `/v3/users/40b7db8901c14cbda81ce095bff44d7f` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:31:59.876635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b7db8901c14cbda81ce095bff44d7f)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:31:59.878613 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:00.162502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053e970b30924e9da928008887ec8273', 'request_id': 'req-8b51590c-229b-448e-af6a-d5e1df1d0754'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.170979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053e970b30924e9da928008887ec8273', 'request_id': 'req-8b51590c-229b-448e-af6a-d5e1df1d0754'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.330800 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b7db8901c14cbda81ce095bff44d7f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.335304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b7db8901c14cbda81ce095bff44d7f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.337154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053e970b30924e9da928008887ec8273', 'request_id': 'req-8b51590c-229b-448e-af6a-d5e1df1d0754'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.339549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b7db8901c14cbda81ce095bff44d7f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.386990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b7db8901c14cbda81ce095bff44d7f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.394037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] The token cache is being invalidate because project 053e970b30924e9da928008887ec8273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:00.394748 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.396485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.399408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 875/1740] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:31:59 2022] DELETE /identity/v3/projects/053e970b30924e9da928008887ec8273 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:00.400729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:00.403152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:00.451771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b7db8901c14cbda81ce095bff44d7f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.580091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51590c-229b-448e-af6a-d5e1df1d0754 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:00.598478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:00.599279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:00.599605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:00.600129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] PATH_INFO: `/v3/projects/3d3a4c4dd27c45b19aed9ab8be3cea93` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:00.602692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d3a4c4dd27c45b19aed9ab8be3cea93)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:00.604852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:00.634566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 866/1741] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:31:59 2022] DELETE /identity/v3/users/40b7db8901c14cbda81ce095bff44d7f => generated 0 bytes in 951 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:00.635668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:00.637951 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:00.694633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:00.859415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3679fbe-269f-4b9e-8ab8-18363c3deadd admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:00.889388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d3a4c4dd27c45b19aed9ab8be3cea93', 'request_id': 'req-3d21bf4b-3da5-42ed-b035-73fbc356acec'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.904956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d3a4c4dd27c45b19aed9ab8be3cea93', 'request_id': 'req-3d21bf4b-3da5-42ed-b035-73fbc356acec'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:00.983156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:00.984221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:00.984498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:00.984772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] PATH_INFO: `/v3/users/b2c78dea998f471296b100b8bfdc1d6b` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:00.988676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c78dea998f471296b100b8bfdc1d6b)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:00.996587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:01.142648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d3a4c4dd27c45b19aed9ab8be3cea93', 'request_id': 'req-3d21bf4b-3da5-42ed-b035-73fbc356acec'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:01.175109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] The token cache is being invalidate because project 3d3a4c4dd27c45b19aed9ab8be3cea93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:01.175589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:01.176632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3d21bf4b-3da5-42ed-b035-73fbc356acec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:01.184048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 876/1742] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:00 2022] DELETE /identity/v3/projects/3d3a4c4dd27c45b19aed9ab8be3cea93 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:01.184048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dffc4-f98b-461b-8c8d-da679bc97062 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:01.184048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dffc4-f98b-461b-8c8d-da679bc97062 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:01.184048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4dffc4-f98b-461b-8c8d-da679bc97062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:01.257952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-eb4dffc4-f98b-461b-8c8d-da679bc97062 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:01.261654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-eb4dffc4-f98b-461b-8c8d-da679bc97062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:01.292584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 877/1743] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:01.295031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6446a-3754-4b7a-bd41-3220ae58bc53 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:01.295647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6446a-3754-4b7a-bd41-3220ae58bc53 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:01.296306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6446a-3754-4b7a-bd41-3220ae58bc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:01.307027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c78dea998f471296b100b8bfdc1d6b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:01.320625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c78dea998f471296b100b8bfdc1d6b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:01.322835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c78dea998f471296b100b8bfdc1d6b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:01.324113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-bed6446a-3754-4b7a-bd41-3220ae58bc53 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:01.325149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bed6446a-3754-4b7a-bd41-3220ae58bc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:01.335214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 878/1744] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:01.346965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a8625-3169-45b0-8e5e-b653028a603d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:01.352295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a8625-3169-45b0-8e5e-b653028a603d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:01.352295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a8625-3169-45b0-8e5e-b653028a603d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:01.383629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c78dea998f471296b100b8bfdc1d6b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:01.405203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-303a8625-3169-45b0-8e5e-b653028a603d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:01.406826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-303a8625-3169-45b0-8e5e-b653028a603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:01.427146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 879/1745] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:01.428294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a8625-3169-45b0-8e5e-b653028a603d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:01.429859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-303a8625-3169-45b0-8e5e-b653028a603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:01.439254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c78dea998f471296b100b8bfdc1d6b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:01.623797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a8625-3169-45b0-8e5e-b653028a603d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:01.635844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c648340-25e1-4d14-ab87-1d9364ea20e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:01.636761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c648340-25e1-4d14-ab87-1d9364ea20e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:01.637090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c648340-25e1-4d14-ab87-1d9364ea20e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:01.637367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c648340-25e1-4d14-ab87-1d9364ea20e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:01.640063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0c648340-25e1-4d14-ab87-1d9364ea20e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:01.682245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 867/1746] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:00 2022] DELETE /identity/v3/users/b2c78dea998f471296b100b8bfdc1d6b => generated 0 bytes in 1047 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:01.684047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:01.729338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c648340-25e1-4d14-ab87-1d9364ea20e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:01.730817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c648340-25e1-4d14-ab87-1d9364ea20e8 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:01.740579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:01.772814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 880/1747] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:01.868084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520517a6-b20b-4e62-a8b4-7bda58e6a6cf admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:01.874877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af6066-02c7-47a6-bf32-399bf06addfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:01.877164 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af6066-02c7-47a6-bf32-399bf06addfb admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:01.877636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af6066-02c7-47a6-bf32-399bf06addfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:01.878029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af6066-02c7-47a6-bf32-399bf06addfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:01.880406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af6066-02c7-47a6-bf32-399bf06addfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:01.880927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af6066-02c7-47a6-bf32-399bf06addfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:01.883278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af6066-02c7-47a6-bf32-399bf06addfb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:01.904238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 868/1748] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:32:01.910174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c648340-25e1-4d14-ab87-1d9364ea20e8 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:01.917278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:01.926031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c648340-25e1-4d14-ab87-1d9364ea20e8 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:01.946268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce38869-d672-4782-9c14-36c120c60cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:01.946899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce38869-d672-4782-9c14-36c120c60cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:01.947168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce38869-d672-4782-9c14-36c120c60cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:01.947440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce38869-d672-4782-9c14-36c120c60cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:01.948061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce38869-d672-4782-9c14-36c120c60cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:01.950015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce38869-d672-4782-9c14-36c120c60cb6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:02.122074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 881/1749] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:01 2022] POST /identity/v3/projects => generated 393 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:02.125219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af6066-02c7-47a6-bf32-399bf06addfb admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:02.151763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af6066-02c7-47a6-bf32-399bf06addfb admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:02.328064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827736e-ba00-4046-beff-b86b2bb87fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:02.330228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827736e-ba00-4046-beff-b86b2bb87fcf admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:02.331700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827736e-ba00-4046-beff-b86b2bb87fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:02.343705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827736e-ba00-4046-beff-b86b2bb87fcf admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:02.347716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827736e-ba00-4046-beff-b86b2bb87fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:02.353962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827736e-ba00-4046-beff-b86b2bb87fcf admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:02.565130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 869/1750] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:02 2022] POST /identity/v3/users => generated 297 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:02.575849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce38869-d672-4782-9c14-36c120c60cb6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:02.606266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce38869-d672-4782-9c14-36c120c60cb6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:02.611806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9da61-4ed1-495c-88e4-5d794edfb2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:02.621536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9da61-4ed1-495c-88e4-5d794edfb2ae admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:02.623170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9da61-4ed1-495c-88e4-5d794edfb2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:02.623814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9da61-4ed1-495c-88e4-5d794edfb2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:02.624696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9da61-4ed1-495c-88e4-5d794edfb2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:02.627728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9da61-4ed1-495c-88e4-5d794edfb2ae admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:02.722232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 882/1751] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:02.726212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827736e-ba00-4046-beff-b86b2bb87fcf admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:02.746530 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827736e-ba00-4046-beff-b86b2bb87fcf admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:02.769251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985099a-7a6b-49b5-a290-796567187f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:02.777714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985099a-7a6b-49b5-a290-796567187f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:02.778129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985099a-7a6b-49b5-a290-796567187f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:02.783083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985099a-7a6b-49b5-a290-796567187f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:02.784247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985099a-7a6b-49b5-a290-796567187f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:02.791164 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985099a-7a6b-49b5-a290-796567187f68 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:02.842306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 870/1752] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:02.848324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9da61-4ed1-495c-88e4-5d794edfb2ae admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:02.870735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9da61-4ed1-495c-88e4-5d794edfb2ae admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:02.893364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92951030-8b68-4edd-97aa-ef90a034d0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:02.894656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92951030-8b68-4edd-97aa-ef90a034d0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:02.895379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92951030-8b68-4edd-97aa-ef90a034d0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:02.895975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92951030-8b68-4edd-97aa-ef90a034d0cc admin admin] PATH_INFO: `/v3/projects/22a11df5c42a423f85fe10c668825b0d/users/f59890c848994c5eb5bde54d9ec7a9b5/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:02.923105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92951030-8b68-4edd-97aa-ef90a034d0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a11df5c42a423f85fe10c668825b0d, user_id=f59890c848994c5eb5bde54d9ec7a9b5, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:02.927510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92951030-8b68-4edd-97aa-ef90a034d0cc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:03.040020 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 883/1753] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:02 2022] PUT /identity/v3/projects/22a11df5c42a423f85fe10c668825b0d/users/f59890c848994c5eb5bde54d9ec7a9b5/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:03.055723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ce299-3811-4fc9-91e6-b6d5bd5d798c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:03.056070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ce299-3811-4fc9-91e6-b6d5bd5d798c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:03.056467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ce299-3811-4fc9-91e6-b6d5bd5d798c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:03.142961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a74ce299-3811-4fc9-91e6-b6d5bd5d798c None None] MFA Rules not processed for user `f59890c848994c5eb5bde54d9ec7a9b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:03.144770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a74ce299-3811-4fc9-91e6-b6d5bd5d798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:03.349325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92951030-8b68-4edd-97aa-ef90a034d0cc admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:03.352837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 871/1754] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:03.504516 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92951030-8b68-4edd-97aa-ef90a034d0cc admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:03.545360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb652e-3725-4d67-89ac-836f25c520b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:03.549461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb652e-3725-4d67-89ac-836f25c520b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:03.549759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb652e-3725-4d67-89ac-836f25c520b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:03.550130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb652e-3725-4d67-89ac-836f25c520b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:03.557036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb652e-3725-4d67-89ac-836f25c520b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:03.559712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb652e-3725-4d67-89ac-836f25c520b7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:03.776276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 884/1755] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:03 2022] POST /identity/v3/projects => generated 391 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:03.779905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ce299-3811-4fc9-91e6-b6d5bd5d798c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:03.813697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ce299-3811-4fc9-91e6-b6d5bd5d798c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:03.864936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69545a2e-4625-438e-983e-5d31ca3e9077 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:03.865274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69545a2e-4625-438e-983e-5d31ca3e9077 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:03.865551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69545a2e-4625-438e-983e-5d31ca3e9077 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:03.867285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 885/1756] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:32:04.030922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5c3f2-e22e-46af-a693-4c0fbd3c0031 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:04.030922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5c3f2-e22e-46af-a693-4c0fbd3c0031 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:04.031217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5c3f2-e22e-46af-a693-4c0fbd3c0031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:04.033745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b43291-4c2d-43a8-8a01-0f44a2cc8164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:04.041958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b43291-4c2d-43a8-8a01-0f44a2cc8164 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:04.042315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b43291-4c2d-43a8-8a01-0f44a2cc8164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:04.042608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b43291-4c2d-43a8-8a01-0f44a2cc8164 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:04.044707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b43291-4c2d-43a8-8a01-0f44a2cc8164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:04.056687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b43291-4c2d-43a8-8a01-0f44a2cc8164 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:04.101714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c9a5c3f2-e22e-46af-a693-4c0fbd3c0031 None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:04.104464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c9a5c3f2-e22e-46af-a693-4c0fbd3c0031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:04.119677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:04.201517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 872/1757] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:03 2022] POST /identity/v3/users => generated 302 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:04.203189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b43291-4c2d-43a8-8a01-0f44a2cc8164 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:04.228082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:04.255691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b43291-4c2d-43a8-8a01-0f44a2cc8164 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:04.270632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4274020-fae4-4424-a8be-3f15890343eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:04.271596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4274020-fae4-4424-a8be-3f15890343eb admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:04.272477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4274020-fae4-4424-a8be-3f15890343eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:04.273012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4274020-fae4-4424-a8be-3f15890343eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:04.273866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4274020-fae4-4424-a8be-3f15890343eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:04.275844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4274020-fae4-4424-a8be-3f15890343eb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:04.312721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 873/1758] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:04.314916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4274020-fae4-4424-a8be-3f15890343eb admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:04.334545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 886/1759] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:32:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:04.343324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4274020-fae4-4424-a8be-3f15890343eb admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:04.356675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cf9ec-f406-45b6-9966-49d3044fb5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:04.356675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cf9ec-f406-45b6-9966-49d3044fb5b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:04.356675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cf9ec-f406-45b6-9966-49d3044fb5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:04.359308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cf9ec-f406-45b6-9966-49d3044fb5b1 admin admin] PATH_INFO: `/v3/projects/1f625c0d6cb842438f8ed9ae72be3826/users/cda1a15139d94b349dbf681c9972348b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:04.402255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24cf9ec-f406-45b6-9966-49d3044fb5b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f625c0d6cb842438f8ed9ae72be3826, user_id=cda1a15139d94b349dbf681c9972348b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:04.406104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24cf9ec-f406-45b6-9966-49d3044fb5b1 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:04.476110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf0b56-6af1-4b4c-8ea4-1bdd30eec651 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:04.476609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf0b56-6af1-4b4c-8ea4-1bdd30eec651 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:04.477089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf0b56-6af1-4b4c-8ea4-1bdd30eec651 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:04.478883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 887/1760] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:32:04.511365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eae05-d118-46dc-ac8d-405842f32dcd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:04.511365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eae05-d118-46dc-ac8d-405842f32dcd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:04.511365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eae05-d118-46dc-ac8d-405842f32dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:04.538776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 874/1761] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:04 2022] PUT /identity/v3/projects/1f625c0d6cb842438f8ed9ae72be3826/users/cda1a15139d94b349dbf681c9972348b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:04.543400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cf9ec-f406-45b6-9966-49d3044fb5b1 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:04.585838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-045eae05-d118-46dc-ac8d-405842f32dcd None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:04.587746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-045eae05-d118-46dc-ac8d-405842f32dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:04.733771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cf9ec-f406-45b6-9966-49d3044fb5b1 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:04.745400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634be2-9258-4f62-a58a-37e6d756551d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:04.761997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634be2-9258-4f62-a58a-37e6d756551d admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:04.762961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634be2-9258-4f62-a58a-37e6d756551d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:04.764047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd634be2-9258-4f62-a58a-37e6d756551d admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:04.764730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634be2-9258-4f62-a58a-37e6d756551d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:04.773369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd634be2-9258-4f62-a58a-37e6d756551d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:04.835763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 875/1762] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:04.841796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634be2-9258-4f62-a58a-37e6d756551d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:04.848928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-dd634be2-9258-4f62-a58a-37e6d756551d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:04.872634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 888/1763] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:32:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:04.872634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eae05-d118-46dc-ac8d-405842f32dcd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:04.903052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd634be2-9258-4f62-a58a-37e6d756551d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:04.913170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163accb-adf1-4687-8a83-f9062a9150fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:04.914272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163accb-adf1-4687-8a83-f9062a9150fe service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:04.916060 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163accb-adf1-4687-8a83-f9062a9150fe service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:04.916721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163accb-adf1-4687-8a83-f9062a9150fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:04.922799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eae05-d118-46dc-ac8d-405842f32dcd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:04.933139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163accb-adf1-4687-8a83-f9062a9150fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:04.935313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163accb-adf1-4687-8a83-f9062a9150fe service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:04.942342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f45a0-ed72-4c45-93c4-43ace35ed82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:04.943368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f45a0-ed72-4c45-93c4-43ace35ed82e admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:04.943848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f45a0-ed72-4c45-93c4-43ace35ed82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:04.946875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f45a0-ed72-4c45-93c4-43ace35ed82e admin admin] PATH_INFO: `/v3/projects/1f625c0d6cb842438f8ed9ae72be3826/users/cda1a15139d94b349dbf681c9972348b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:04.951925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 876/1764] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:04.954206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f45a0-ed72-4c45-93c4-43ace35ed82e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f625c0d6cb842438f8ed9ae72be3826, user_id=cda1a15139d94b349dbf681c9972348b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:04.964419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f45a0-ed72-4c45-93c4-43ace35ed82e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:05.033626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 889/1765] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:04 2022] PUT /identity/v3/projects/1f625c0d6cb842438f8ed9ae72be3826/users/cda1a15139d94b349dbf681c9972348b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:05.045870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee112f-5e65-4be4-9f71-a3ffe1b714a9 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:05.046754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee112f-5e65-4be4-9f71-a3ffe1b714a9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:05.048684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee112f-5e65-4be4-9f71-a3ffe1b714a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:05.098710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f45a0-ed72-4c45-93c4-43ace35ed82e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:05.100479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bf6f45a0-ed72-4c45-93c4-43ace35ed82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:05.156580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a6ee112f-5e65-4be4-9f71-a3ffe1b714a9 None None] MFA Rules not processed for user `cda1a15139d94b349dbf681c9972348b`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:05.171816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a6ee112f-5e65-4be4-9f71-a3ffe1b714a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:05.295111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f45a0-ed72-4c45-93c4-43ace35ed82e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:05.313182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:05.314050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:05.314499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:05.314918 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] PATH_INFO: `/v3/projects/ccdea3274248427f947b59b76f1a54f8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:05.320218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccdea3274248427f947b59b76f1a54f8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:05.322739 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:05.387444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 877/1766] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:05 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:05.438772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee112f-5e65-4be4-9f71-a3ffe1b714a9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:05.440780 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a6ee112f-5e65-4be4-9f71-a3ffe1b714a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:05.547014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccdea3274248427f947b59b76f1a54f8', 'request_id': 'req-3aab91c3-74fc-4e10-a632-40489c0f8053'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:05.581067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccdea3274248427f947b59b76f1a54f8', 'request_id': 'req-3aab91c3-74fc-4e10-a632-40489c0f8053'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:05.624829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee112f-5e65-4be4-9f71-a3ffe1b714a9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:05.655167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccdea3274248427f947b59b76f1a54f8', 'request_id': 'req-3aab91c3-74fc-4e10-a632-40489c0f8053'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:05.717292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] The token cache is being invalidate because project ccdea3274248427f947b59b76f1a54f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:05.718527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:05.720949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:05.725941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 890/1767] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:05 2022] DELETE /identity/v3/projects/ccdea3274248427f947b59b76f1a54f8 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:32:05.727165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:05.732402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab3447-3bb2-479d-9b3e-0e618ea9a6f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:05.732654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:05.734433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3447-3bb2-479d-9b3e-0e618ea9a6f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:05.735608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3447-3bb2-479d-9b3e-0e618ea9a6f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:05.737354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3447-3bb2-479d-9b3e-0e618ea9a6f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:05.762620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6eab3447-3bb2-479d-9b3e-0e618ea9a6f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:06.006995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab3447-3bb2-479d-9b3e-0e618ea9a6f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:06.011531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab3447-3bb2-479d-9b3e-0e618ea9a6f3 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:06.015624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab91c3-74fc-4e10-a632-40489c0f8053 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:06.026271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:06.026940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:06.027313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:06.027572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:06.030083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:06.045280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 878/1768] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:05 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:06.046703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf6db4-e121-4f66-b55c-b955adffa8ca None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:06.046990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf6db4-e121-4f66-b55c-b955adffa8ca None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:06.047411 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf6db4-e121-4f66-b55c-b955adffa8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:06.049707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:06.051555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:06.068229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 891/1769] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:06.115871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-63cf6db4-e121-4f66-b55c-b955adffa8ca None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:06.121319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-63cf6db4-e121-4f66-b55c-b955adffa8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:06.176291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:06.178191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:06.207657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b396af-2d8a-49f3-bb40-0d92dc9840d4 service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:06.225095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61b116-e6fc-4b37-a4d9-f42a32f32ad0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:06.230115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61b116-e6fc-4b37-a4d9-f42a32f32ad0 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:06.232135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61b116-e6fc-4b37-a4d9-f42a32f32ad0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:06.232463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61b116-e6fc-4b37-a4d9-f42a32f32ad0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:06.241733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dd61b116-e6fc-4b37-a4d9-f42a32f32ad0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:06.332985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 879/1770] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:06.341477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabfffa-5269-43d0-97f6-a3dcad87505a None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:06.344399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabfffa-5269-43d0-97f6-a3dcad87505a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:06.344399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabfffa-5269-43d0-97f6-a3dcad87505a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:06.407616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ceabfffa-5269-43d0-97f6-a3dcad87505a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:06.408898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ceabfffa-5269-43d0-97f6-a3dcad87505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:06.409265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61b116-e6fc-4b37-a4d9-f42a32f32ad0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:06.413140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61b116-e6fc-4b37-a4d9-f42a32f32ad0 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:06.428379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:06.449180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 880/1771] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:06.450633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabfffa-5269-43d0-97f6-a3dcad87505a None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:06.467403 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 892/1772] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:06 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:32:06.468427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61b116-e6fc-4b37-a4d9-f42a32f32ad0 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:06.477148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ceabfffa-5269-43d0-97f6-a3dcad87505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:06.507023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61b116-e6fc-4b37-a4d9-f42a32f32ad0 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:06.510504 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabfffa-5269-43d0-97f6-a3dcad87505a None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:06.519484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c4205-64eb-4355-ac3c-da8b09a65f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:06.520112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c4205-64eb-4355-ac3c-da8b09a65f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:06.521333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c4205-64eb-4355-ac3c-da8b09a65f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:06.521333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c4205-64eb-4355-ac3c-da8b09a65f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:06.521333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c4205-64eb-4355-ac3c-da8b09a65f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:06.521621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c4205-64eb-4355-ac3c-da8b09a65f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:06.526884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c4205-64eb-4355-ac3c-da8b09a65f18 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:06.538632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:06.539526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:06.539944 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:06.540411 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] PATH_INFO: `/v3/projects/472bfc5a99a946c796dc0425f757d497` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:06.542619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=472bfc5a99a946c796dc0425f757d497)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:06.556379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:06.560657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 893/1773] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:32:06.566714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c4205-64eb-4355-ac3c-da8b09a65f18 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:06.580863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:06.591962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c4205-64eb-4355-ac3c-da8b09a65f18 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:06.597784 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ce8a4-41fa-4583-bbcf-80289253650a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:06.598470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ce8a4-41fa-4583-bbcf-80289253650a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:06.599242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ce8a4-41fa-4583-bbcf-80289253650a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:06.599531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ce8a4-41fa-4583-bbcf-80289253650a admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:06.600269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ce8a4-41fa-4583-bbcf-80289253650a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:06.618824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ce8a4-41fa-4583-bbcf-80289253650a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:06.756993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 894/1774] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:06 2022] POST /identity/v3/projects => generated 405 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:06.761411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ce8a4-41fa-4583-bbcf-80289253650a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:06.829203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '472bfc5a99a946c796dc0425f757d497', 'request_id': 'req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:06.859351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '472bfc5a99a946c796dc0425f757d497', 'request_id': 'req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:06.900778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ce8a4-41fa-4583-bbcf-80289253650a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:06.958753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '472bfc5a99a946c796dc0425f757d497', 'request_id': 'req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:07.017107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] The token cache is being invalidate because project 472bfc5a99a946c796dc0425f757d497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:07.017107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:07.017107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:07.018193 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 881/1775] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:06 2022] DELETE /identity/v3/projects/472bfc5a99a946c796dc0425f757d497 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:07.025156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:07.028586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:07.166433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574aecd-e476-498b-b1c0-5a7211a4bb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:07.167063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574aecd-e476-498b-b1c0-5a7211a4bb75 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:07.167325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574aecd-e476-498b-b1c0-5a7211a4bb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:07.167595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574aecd-e476-498b-b1c0-5a7211a4bb75 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:07.168416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574aecd-e476-498b-b1c0-5a7211a4bb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:07.180919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574aecd-e476-498b-b1c0-5a7211a4bb75 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:07.285255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a86e8d-918b-4495-87b0-cd7408ebf3d5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:07.294831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:07.295636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:07.295923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:07.296213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] PATH_INFO: `/v3/users/614cf723b22d48f7a057744e34c90f25` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:07.298163 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=614cf723b22d48f7a057744e34c90f25)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:07.300125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:08.038123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 895/1776] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:06 2022] POST /identity/v3/users => generated 303 bytes in 1278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:32:08.098205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574aecd-e476-498b-b1c0-5a7211a4bb75 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:08.100125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d574aecd-e476-498b-b1c0-5a7211a4bb75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:08.114762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574aecd-e476-498b-b1c0-5a7211a4bb75 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:08.121437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e7988-1c96-416e-a90c-05563fd3b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:08.122067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e7988-1c96-416e-a90c-05563fd3b1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:08.122343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e7988-1c96-416e-a90c-05563fd3b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:08.122662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e7988-1c96-416e-a90c-05563fd3b1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:08.123316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e7988-1c96-416e-a90c-05563fd3b1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:08.125904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e7988-1c96-416e-a90c-05563fd3b1d8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:08.140874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 896/1777] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:08.146255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e7988-1c96-416e-a90c-05563fd3b1d8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:08.163084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e7988-1c96-416e-a90c-05563fd3b1d8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:08.171205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252762dc-e08b-4065-a0c6-88fe539af762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:08.171205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252762dc-e08b-4065-a0c6-88fe539af762 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:08.171941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252762dc-e08b-4065-a0c6-88fe539af762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:08.171941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252762dc-e08b-4065-a0c6-88fe539af762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:08.173138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252762dc-e08b-4065-a0c6-88fe539af762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:08.174965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252762dc-e08b-4065-a0c6-88fe539af762 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:08.195164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 897/1778] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:08.199201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252762dc-e08b-4065-a0c6-88fe539af762 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:08.225526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252762dc-e08b-4065-a0c6-88fe539af762 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:08.256251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661daa8-cb9d-4048-b430-95b3f88a7c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:08.261663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661daa8-cb9d-4048-b430-95b3f88a7c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:08.272719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661daa8-cb9d-4048-b430-95b3f88a7c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:08.273127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661daa8-cb9d-4048-b430-95b3f88a7c9c admin admin] PATH_INFO: `/v3/projects/de2dd5c0338541d68355d679b053bbf0/users/a3baf00ac52144bf82e01c0e335a5fd3/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:08.314152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661daa8-cb9d-4048-b430-95b3f88a7c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2dd5c0338541d68355d679b053bbf0, user_id=a3baf00ac52144bf82e01c0e335a5fd3, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:08.318088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661daa8-cb9d-4048-b430-95b3f88a7c9c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:08.981564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 898/1779] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:08 2022] PUT /identity/v3/projects/de2dd5c0338541d68355d679b053bbf0/users/a3baf00ac52144bf82e01c0e335a5fd3/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:08.989944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684c243-4953-45da-926e-ac8acb9e8688 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:08.990369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684c243-4953-45da-926e-ac8acb9e8688 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:08.990782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684c243-4953-45da-926e-ac8acb9e8688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:09.042584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '614cf723b22d48f7a057744e34c90f25'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:09.045305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '614cf723b22d48f7a057744e34c90f25'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:09.046515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '614cf723b22d48f7a057744e34c90f25'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:09.073380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1684c243-4953-45da-926e-ac8acb9e8688 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:09.075141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1684c243-4953-45da-926e-ac8acb9e8688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:09.082515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '614cf723b22d48f7a057744e34c90f25'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:09.119855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '614cf723b22d48f7a057744e34c90f25'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:09.284080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 899/1780] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:09.301189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684c243-4953-45da-926e-ac8acb9e8688 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:09.303577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1684c243-4953-45da-926e-ac8acb9e8688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:09.341317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 882/1781] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:07 2022] DELETE /identity/v3/users/614cf723b22d48f7a057744e34c90f25 => generated 0 bytes in 2318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:09.473105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684c243-4953-45da-926e-ac8acb9e8688 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:09.718719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028a0dd-3be2-4e52-bb02-41f49edb6357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:09.718719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028a0dd-3be2-4e52-bb02-41f49edb6357 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:09.718719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028a0dd-3be2-4e52-bb02-41f49edb6357 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:09.718719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028a0dd-3be2-4e52-bb02-41f49edb6357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:09.944759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028a0dd-3be2-4e52-bb02-41f49edb6357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:09.946063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028a0dd-3be2-4e52-bb02-41f49edb6357 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:09.970883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 900/1782] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:10.974690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:10.976941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:10.991962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87e206-1b16-4e55-a435-dcdcf55be919 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:10.998090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:10.998870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:10.999151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:10.999421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] PATH_INFO: `/v3/projects/77ef209fe2d44d499ee10bdbc8000a19` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:11.000730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ef209fe2d44d499ee10bdbc8000a19)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:11.011275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:11.278162 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ef209fe2d44d499ee10bdbc8000a19', 'request_id': 'req-da2bd9de-f15b-4b2a-9095-fe11799c716c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:11.284537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ef209fe2d44d499ee10bdbc8000a19', 'request_id': 'req-da2bd9de-f15b-4b2a-9095-fe11799c716c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:11.409615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ef209fe2d44d499ee10bdbc8000a19', 'request_id': 'req-da2bd9de-f15b-4b2a-9095-fe11799c716c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:11.424892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] The token cache is being invalidate because project 77ef209fe2d44d499ee10bdbc8000a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:11.425276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:11.426036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-da2bd9de-f15b-4b2a-9095-fe11799c716c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:11.427701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 883/1783] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:10 2022] DELETE /identity/v3/projects/77ef209fe2d44d499ee10bdbc8000a19 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:11.433712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0f43e-fc67-40da-853b-705547d7116c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:11.434349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0f43e-fc67-40da-853b-705547d7116c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:11.434349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0f43e-fc67-40da-853b-705547d7116c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:11.465188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-64a0f43e-fc67-40da-853b-705547d7116c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:11.466679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-64a0f43e-fc67-40da-853b-705547d7116c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:11.545518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 901/1784] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:11.549833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fef42a-4114-4127-a6b6-30a66d03add5 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:11.550170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fef42a-4114-4127-a6b6-30a66d03add5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:11.550444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fef42a-4114-4127-a6b6-30a66d03add5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:11.572359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-48fef42a-4114-4127-a6b6-30a66d03add5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:11.573799 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-48fef42a-4114-4127-a6b6-30a66d03add5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:11.583307 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 884/1785] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:11.600093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0f43e-fc67-40da-853b-705547d7116c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:11.612317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0f43e-fc67-40da-853b-705547d7116c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:11.617303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95815f-3749-4b6e-8ffb-0c1a1f6fea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:11.617902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95815f-3749-4b6e-8ffb-0c1a1f6fea83 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:11.618179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95815f-3749-4b6e-8ffb-0c1a1f6fea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:11.618447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95815f-3749-4b6e-8ffb-0c1a1f6fea83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:11.619589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95815f-3749-4b6e-8ffb-0c1a1f6fea83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:11.619894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95815f-3749-4b6e-8ffb-0c1a1f6fea83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:11.621590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95815f-3749-4b6e-8ffb-0c1a1f6fea83 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:11.634297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 902/1786] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:32:11.638630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fef42a-4114-4127-a6b6-30a66d03add5 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:11.651903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fef42a-4114-4127-a6b6-30a66d03add5 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:11.653348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3034f-0387-4481-8b02-283083568815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:11.654148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3034f-0387-4481-8b02-283083568815 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:11.654586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3034f-0387-4481-8b02-283083568815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:11.655022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3034f-0387-4481-8b02-283083568815 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:11.655659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3034f-0387-4481-8b02-283083568815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:11.657880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3034f-0387-4481-8b02-283083568815 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:11.835543 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 885/1787] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:11 2022] POST /identity/v3/projects => generated 381 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:11.839237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95815f-3749-4b6e-8ffb-0c1a1f6fea83 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:11.851583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95815f-3749-4b6e-8ffb-0c1a1f6fea83 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:11.913028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee59127-ac5e-4c38-9ec8-44a58e6a16ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:11.913725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee59127-ac5e-4c38-9ec8-44a58e6a16ea admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:11.914006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee59127-ac5e-4c38-9ec8-44a58e6a16ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:11.914280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee59127-ac5e-4c38-9ec8-44a58e6a16ea admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:11.914901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee59127-ac5e-4c38-9ec8-44a58e6a16ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:11.916919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee59127-ac5e-4c38-9ec8-44a58e6a16ea admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:12.070511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 903/1788] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:11 2022] POST /identity/v3/users => generated 291 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:12.074820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3034f-0387-4481-8b02-283083568815 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:12.086547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3034f-0387-4481-8b02-283083568815 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:12.096458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b63208-b248-439c-b25e-9b859ced77da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:12.097172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b63208-b248-439c-b25e-9b859ced77da admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:12.097518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b63208-b248-439c-b25e-9b859ced77da admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:12.097736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b63208-b248-439c-b25e-9b859ced77da admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:12.098440 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b63208-b248-439c-b25e-9b859ced77da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:12.100221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b63208-b248-439c-b25e-9b859ced77da admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:12.113470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 886/1789] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:32:12.118031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee59127-ac5e-4c38-9ec8-44a58e6a16ea admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:12.131065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee59127-ac5e-4c38-9ec8-44a58e6a16ea admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:12.138434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505179ee-1d07-4302-86d3-8f0e7af5dd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:12.149957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505179ee-1d07-4302-86d3-8f0e7af5dd1a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:12.149957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505179ee-1d07-4302-86d3-8f0e7af5dd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:12.149957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505179ee-1d07-4302-86d3-8f0e7af5dd1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:12.149957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505179ee-1d07-4302-86d3-8f0e7af5dd1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:12.149957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505179ee-1d07-4302-86d3-8f0e7af5dd1a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:12.160849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 904/1790] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:12.162437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b63208-b248-439c-b25e-9b859ced77da admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:12.173196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b63208-b248-439c-b25e-9b859ced77da admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:12.177746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c22c23-c3c3-43c4-beb1-8c8f2acf9090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:12.178393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c22c23-c3c3-43c4-beb1-8c8f2acf9090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:12.178670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c22c23-c3c3-43c4-beb1-8c8f2acf9090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:12.178942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c22c23-c3c3-43c4-beb1-8c8f2acf9090 admin admin] PATH_INFO: `/v3/projects/1a77a392285345bc8754c13896169e1b/users/ffbb18827cde471eb1a37dd386e07e29/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:12.194384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c22c23-c3c3-43c4-beb1-8c8f2acf9090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a77a392285345bc8754c13896169e1b, user_id=ffbb18827cde471eb1a37dd386e07e29, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:12.197122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c22c23-c3c3-43c4-beb1-8c8f2acf9090 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:12.330674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 887/1791] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:12 2022] PUT /identity/v3/projects/1a77a392285345bc8754c13896169e1b/users/ffbb18827cde471eb1a37dd386e07e29/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:12.338276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4fc4e-8380-4172-a199-2021330b953d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:12.338564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4fc4e-8380-4172-a199-2021330b953d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:12.338839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4fc4e-8380-4172-a199-2021330b953d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:12.368596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d5d4fc4e-8380-4172-a199-2021330b953d None None] MFA Rules not processed for user `ffbb18827cde471eb1a37dd386e07e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:12.370025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d5d4fc4e-8380-4172-a199-2021330b953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:12.445542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 905/1792] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:12.450447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c22c23-c3c3-43c4-beb1-8c8f2acf9090 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:12.548130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c22c23-c3c3-43c4-beb1-8c8f2acf9090 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:12.553148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16664fde-b32a-4ee4-bafc-497f0582d021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:12.553835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16664fde-b32a-4ee4-bafc-497f0582d021 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:12.554184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16664fde-b32a-4ee4-bafc-497f0582d021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:12.554525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16664fde-b32a-4ee4-bafc-497f0582d021 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:12.555276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16664fde-b32a-4ee4-bafc-497f0582d021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:12.557383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16664fde-b32a-4ee4-bafc-497f0582d021 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:12.661013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349f87c-3647-4af9-b301-cffa108b9f8f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:12.661399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349f87c-3647-4af9-b301-cffa108b9f8f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:12.661681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349f87c-3647-4af9-b301-cffa108b9f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:12.667867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:12.717024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f349f87c-3647-4af9-b301-cffa108b9f8f None None] MFA Rules not processed for user `a3baf00ac52144bf82e01c0e335a5fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:12.718413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f349f87c-3647-4af9-b301-cffa108b9f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:12.794924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 906/1793] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:12 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:12.800213 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349f87c-3647-4af9-b301-cffa108b9f8f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:12.801845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f349f87c-3647-4af9-b301-cffa108b9f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:12.802973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 888/1794] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:12 2022] POST /identity/v3/projects => generated 379 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:12.805230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16664fde-b32a-4ee4-bafc-497f0582d021 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:12.816833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349f87c-3647-4af9-b301-cffa108b9f8f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:12.896347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873957a-ed4d-452f-a2e7-849ab61d7d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:12.897175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873957a-ed4d-452f-a2e7-849ab61d7d79 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:12.897547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873957a-ed4d-452f-a2e7-849ab61d7d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:12.897889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16664fde-b32a-4ee4-bafc-497f0582d021 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:12.898015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873957a-ed4d-452f-a2e7-849ab61d7d79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:12.898743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873957a-ed4d-452f-a2e7-849ab61d7d79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:12.900897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873957a-ed4d-452f-a2e7-849ab61d7d79 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:12.903085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae24799-f47b-4e91-866d-7833673771b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:12.904212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae24799-f47b-4e91-866d-7833673771b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:12.904212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae24799-f47b-4e91-866d-7833673771b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:12.904416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae24799-f47b-4e91-866d-7833673771b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:12.905129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae24799-f47b-4e91-866d-7833673771b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:12.908082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae24799-f47b-4e91-866d-7833673771b2 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:13.063591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 907/1795] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:12 2022] POST /identity/v3/projects => generated 403 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:32:13.064114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 889/1796] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:12 2022] POST /identity/v3/users => generated 296 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:32:13.069855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873957a-ed4d-452f-a2e7-849ab61d7d79 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:13.071175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae24799-f47b-4e91-866d-7833673771b2 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:13.099520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae24799-f47b-4e91-866d-7833673771b2 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:13.177523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:13.195537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873957a-ed4d-452f-a2e7-849ab61d7d79 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:13.202639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b08e78-295a-4b54-874a-3d2fbf0deb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:13.204367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b08e78-295a-4b54-874a-3d2fbf0deb2d admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:13.204883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b08e78-295a-4b54-874a-3d2fbf0deb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:13.205534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51cf05-1207-497a-bcb0-bbcafa20c31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:13.206064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b08e78-295a-4b54-874a-3d2fbf0deb2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:13.206611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51cf05-1207-497a-bcb0-bbcafa20c31d admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:13.207570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b08e78-295a-4b54-874a-3d2fbf0deb2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:13.209847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b08e78-295a-4b54-874a-3d2fbf0deb2d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:13.211540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51cf05-1207-497a-bcb0-bbcafa20c31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:13.213842 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51cf05-1207-497a-bcb0-bbcafa20c31d admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:13.215311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51cf05-1207-497a-bcb0-bbcafa20c31d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:13.226940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51cf05-1207-497a-bcb0-bbcafa20c31d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:13.231414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 908/1797] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:13.237837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b08e78-295a-4b54-874a-3d2fbf0deb2d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:13.257247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b08e78-295a-4b54-874a-3d2fbf0deb2d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:13.265200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeac86d-c6ff-4c56-b414-8b08da6a9715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:13.267679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeac86d-c6ff-4c56-b414-8b08da6a9715 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:13.268034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeac86d-c6ff-4c56-b414-8b08da6a9715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:13.268320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeac86d-c6ff-4c56-b414-8b08da6a9715 admin admin] PATH_INFO: `/v3/projects/7ba8fcb67877424a8348c0cdf7ab62f0/users/ff31c5bf56b643ab8562b0fb5fea2000/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:13.289961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeac86d-c6ff-4c56-b414-8b08da6a9715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba8fcb67877424a8348c0cdf7ab62f0, user_id=ff31c5bf56b643ab8562b0fb5fea2000, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:13.293972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeac86d-c6ff-4c56-b414-8b08da6a9715 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:13.407111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 890/1798] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:13 2022] POST /identity/v3/users => generated 308 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:13.407111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51cf05-1207-497a-bcb0-bbcafa20c31d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:13.418357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51cf05-1207-497a-bcb0-bbcafa20c31d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:13.425073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338064a-b165-4c85-9669-299d441fab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:13.425660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338064a-b165-4c85-9669-299d441fab67 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:13.425941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338064a-b165-4c85-9669-299d441fab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:13.426194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338064a-b165-4c85-9669-299d441fab67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:13.427576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338064a-b165-4c85-9669-299d441fab67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:13.429358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338064a-b165-4c85-9669-299d441fab67 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:13.446373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 891/1799] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:32:13.452321 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338064a-b165-4c85-9669-299d441fab67 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:13.482504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 909/1800] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:13 2022] PUT /identity/v3/projects/7ba8fcb67877424a8348c0cdf7ab62f0/users/ff31c5bf56b643ab8562b0fb5fea2000/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:13.487627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeac86d-c6ff-4c56-b414-8b08da6a9715 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:13.489537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338064a-b165-4c85-9669-299d441fab67 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:13.636442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ead44b-edf1-40fe-a57a-385dcdd16d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:13.642174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead44b-edf1-40fe-a57a-385dcdd16d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:13.642174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead44b-edf1-40fe-a57a-385dcdd16d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:13.642174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead44b-edf1-40fe-a57a-385dcdd16d04 admin admin] PATH_INFO: `/v3/projects/36c8c75d1f364fe3aceb9f0577514756/users/eb6dfc33a8d348fab9335840a9d67bd1/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:13.645693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeac86d-c6ff-4c56-b414-8b08da6a9715 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:13.657269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ef23c-e80c-4335-adbe-f95f0e9b83bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:13.658394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ef23c-e80c-4335-adbe-f95f0e9b83bb admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:13.659061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ef23c-e80c-4335-adbe-f95f0e9b83bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:13.659505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ef23c-e80c-4335-adbe-f95f0e9b83bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:13.660453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ef23c-e80c-4335-adbe-f95f0e9b83bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:13.662520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ef23c-e80c-4335-adbe-f95f0e9b83bb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:13.672114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ead44b-edf1-40fe-a57a-385dcdd16d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c8c75d1f364fe3aceb9f0577514756, user_id=eb6dfc33a8d348fab9335840a9d67bd1, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:13.675714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ead44b-edf1-40fe-a57a-385dcdd16d04 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:13.681254 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 910/1801] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:32:13.683982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ef23c-e80c-4335-adbe-f95f0e9b83bb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:13.702711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ef23c-e80c-4335-adbe-f95f0e9b83bb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:13.710337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da310b23-bbc4-496b-b363-85d73a133e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:13.711175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da310b23-bbc4-496b-b363-85d73a133e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:13.711600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da310b23-bbc4-496b-b363-85d73a133e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:13.712079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da310b23-bbc4-496b-b363-85d73a133e8e admin admin] PATH_INFO: `/v3/projects/7ba8fcb67877424a8348c0cdf7ab62f0/users/ff31c5bf56b643ab8562b0fb5fea2000/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:13.715432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da310b23-bbc4-496b-b363-85d73a133e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba8fcb67877424a8348c0cdf7ab62f0, user_id=ff31c5bf56b643ab8562b0fb5fea2000, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:13.720326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da310b23-bbc4-496b-b363-85d73a133e8e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:13.737817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 911/1802] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:13 2022] PUT /identity/v3/projects/7ba8fcb67877424a8348c0cdf7ab62f0/users/ff31c5bf56b643ab8562b0fb5fea2000/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:13.755598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22313268-4c7f-4703-b34e-7e79263f1019 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:13.756094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22313268-4c7f-4703-b34e-7e79263f1019 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:13.756467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22313268-4c7f-4703-b34e-7e79263f1019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:13.809077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-22313268-4c7f-4703-b34e-7e79263f1019 None None] MFA Rules not processed for user `ff31c5bf56b643ab8562b0fb5fea2000`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:13.814928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-22313268-4c7f-4703-b34e-7e79263f1019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:13.821191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 892/1803] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:13 2022] PUT /identity/v3/projects/36c8c75d1f364fe3aceb9f0577514756/users/eb6dfc33a8d348fab9335840a9d67bd1/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:13.825932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ead44b-edf1-40fe-a57a-385dcdd16d04 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:13.941661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ead44b-edf1-40fe-a57a-385dcdd16d04 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:13.945780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 912/1804] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:13.949277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de57630-bb99-4234-a4c2-cb19c1f22925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:13.950166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de57630-bb99-4234-a4c2-cb19c1f22925 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:13.950694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de57630-bb99-4234-a4c2-cb19c1f22925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:13.951448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de57630-bb99-4234-a4c2-cb19c1f22925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:13.951988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de57630-bb99-4234-a4c2-cb19c1f22925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:13.954141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de57630-bb99-4234-a4c2-cb19c1f22925 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:13.958106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22313268-4c7f-4703-b34e-7e79263f1019 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:13.963778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-22313268-4c7f-4703-b34e-7e79263f1019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:13.987302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 893/1805] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:32:13.991640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de57630-bb99-4234-a4c2-cb19c1f22925 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:14.007384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de57630-bb99-4234-a4c2-cb19c1f22925 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:14.014907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54b9b0-5e69-4a70-840b-c0ac0a6369ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:14.015584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54b9b0-5e69-4a70-840b-c0ac0a6369ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:14.026182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54b9b0-5e69-4a70-840b-c0ac0a6369ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:14.026182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54b9b0-5e69-4a70-840b-c0ac0a6369ee admin admin] PATH_INFO: `/v3/projects/36c8c75d1f364fe3aceb9f0577514756/users/eb6dfc33a8d348fab9335840a9d67bd1/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:14.026182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54b9b0-5e69-4a70-840b-c0ac0a6369ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c8c75d1f364fe3aceb9f0577514756, user_id=eb6dfc33a8d348fab9335840a9d67bd1, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:14.026182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db54b9b0-5e69-4a70-840b-c0ac0a6369ee admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:14.045118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 894/1806] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:13 2022] PUT /identity/v3/projects/36c8c75d1f364fe3aceb9f0577514756/users/eb6dfc33a8d348fab9335840a9d67bd1/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:14.060623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22313268-4c7f-4703-b34e-7e79263f1019 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:14.145379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5187a-894d-47df-87d5-a23a979aeba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:14.146340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5187a-894d-47df-87d5-a23a979aeba2 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:14.147141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5187a-894d-47df-87d5-a23a979aeba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:14.147682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5187a-894d-47df-87d5-a23a979aeba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:14.280336 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5187a-894d-47df-87d5-a23a979aeba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:14.287291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5187a-894d-47df-87d5-a23a979aeba2 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:14.294533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 913/1807] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:15.200721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54b9b0-5e69-4a70-840b-c0ac0a6369ee admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:15.275035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54b9b0-5e69-4a70-840b-c0ac0a6369ee admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:15.286728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:15.287391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:15.287669 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:15.287942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] PATH_INFO: `/v3/users/ffbb18827cde471eb1a37dd386e07e29` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:15.289209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffbb18827cde471eb1a37dd386e07e29)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:15.291206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:15.525649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffbb18827cde471eb1a37dd386e07e29'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:15.526159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffbb18827cde471eb1a37dd386e07e29'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:15.526671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffbb18827cde471eb1a37dd386e07e29'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:15.545794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffbb18827cde471eb1a37dd386e07e29'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:15.564872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffbb18827cde471eb1a37dd386e07e29'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:15.652300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 895/1808] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:15 2022] DELETE /identity/v3/users/ffbb18827cde471eb1a37dd386e07e29 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:15.657808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22d989-08b6-4197-b1b0-7b43a839a825 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:15.658215 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22d989-08b6-4197-b1b0-7b43a839a825 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:15.658447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22d989-08b6-4197-b1b0-7b43a839a825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:15.681969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5f22d989-08b6-4197-b1b0-7b43a839a825 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:15.683223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5f22d989-08b6-4197-b1b0-7b43a839a825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:15.749724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 914/1809] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:15.756497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:15.761259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:15.838912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ee5ab-6c29-423d-95c3-6a47ba506a79 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:15.842977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296123c7-9158-4a71-bd79-d2717259c5f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:15.843599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296123c7-9158-4a71-bd79-d2717259c5f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:15.844041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296123c7-9158-4a71-bd79-d2717259c5f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:15.844904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296123c7-9158-4a71-bd79-d2717259c5f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:15.854372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296123c7-9158-4a71-bd79-d2717259c5f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:15.855807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296123c7-9158-4a71-bd79-d2717259c5f5 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:15.865919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 896/1810] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:16.170369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22d989-08b6-4197-b1b0-7b43a839a825 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:16.172716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5f22d989-08b6-4197-b1b0-7b43a839a825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:16.186504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22d989-08b6-4197-b1b0-7b43a839a825 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:16.194464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:16.195168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:16.195489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:16.195790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] PATH_INFO: `/v3/users/f59890c848994c5eb5bde54d9ec7a9b5` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:16.197042 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f59890c848994c5eb5bde54d9ec7a9b5)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:16.199771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:16.229269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:16.467725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296123c7-9158-4a71-bd79-d2717259c5f5 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:16.469813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-296123c7-9158-4a71-bd79-d2717259c5f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:16.481402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f59890c848994c5eb5bde54d9ec7a9b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.489352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296123c7-9158-4a71-bd79-d2717259c5f5 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:16.498685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:16.498685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:16.498685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:16.498685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] PATH_INFO: `/v3/projects/1a77a392285345bc8754c13896169e1b` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:16.499216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a77a392285345bc8754c13896169e1b)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:16.499900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:16.592968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f59890c848994c5eb5bde54d9ec7a9b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.608813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f59890c848994c5eb5bde54d9ec7a9b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.609725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f59890c848994c5eb5bde54d9ec7a9b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.617948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0522ae8f-87e2-4dfb-bc1e-cb23d4c16ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f59890c848994c5eb5bde54d9ec7a9b5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.630285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 915/1811] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:16 2022] DELETE /identity/v3/users/f59890c848994c5eb5bde54d9ec7a9b5 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:16.635326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f55cb-aaf5-4a9e-ac80-ddf71298c73b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:16.635714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f55cb-aaf5-4a9e-ac80-ddf71298c73b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:16.636089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f55cb-aaf5-4a9e-ac80-ddf71298c73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:16.668463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a93f55cb-aaf5-4a9e-ac80-ddf71298c73b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:16.671182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a93f55cb-aaf5-4a9e-ac80-ddf71298c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:16.680891 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a77a392285345bc8754c13896169e1b', 'request_id': 'req-88d2ad67-41c8-4a72-820b-f78f792c10e4'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.689020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a77a392285345bc8754c13896169e1b', 'request_id': 'req-88d2ad67-41c8-4a72-820b-f78f792c10e4'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.785909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 916/1812] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:16.787363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd59bf-5ed9-464d-a24a-4fe78ab6eb25 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:16.787823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd59bf-5ed9-464d-a24a-4fe78ab6eb25 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:16.789182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd59bf-5ed9-464d-a24a-4fe78ab6eb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:16.830791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a2cd59bf-5ed9-464d-a24a-4fe78ab6eb25 None None] MFA Rules not processed for user `eb6dfc33a8d348fab9335840a9d67bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:16.833800 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a77a392285345bc8754c13896169e1b', 'request_id': 'req-88d2ad67-41c8-4a72-820b-f78f792c10e4'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.833909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a2cd59bf-5ed9-464d-a24a-4fe78ab6eb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:16.849762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] The token cache is being invalidate because project 1a77a392285345bc8754c13896169e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:16.850368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.851250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:16.853015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 897/1813] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:16 2022] DELETE /identity/v3/projects/1a77a392285345bc8754c13896169e1b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:32:16.853999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:16.855561 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:16.921761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2ad67-41c8-4a72-820b-f78f792c10e4 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:16.926134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:16.926743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:16.927016 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:16.927283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:16.928103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 917/1814] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:16 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:16.929406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd59bf-5ed9-464d-a24a-4fe78ab6eb25 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:16.929800 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:16.933351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a2cd59bf-5ed9-464d-a24a-4fe78ab6eb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:16.942919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:16.945215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:16.949270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd59bf-5ed9-464d-a24a-4fe78ab6eb25 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:16.957567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:16.957984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 898/1815] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:16.958839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:16.959125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:16.959400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] PATH_INFO: `/v3/users/ff31c5bf56b643ab8562b0fb5fea2000` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:16.960662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff31c5bf56b643ab8562b0fb5fea2000)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:16.962629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:17.347684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff31c5bf56b643ab8562b0fb5fea2000'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:17.485400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff31c5bf56b643ab8562b0fb5fea2000'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:17.500653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff31c5bf56b643ab8562b0fb5fea2000'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:17.502849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff31c5bf56b643ab8562b0fb5fea2000'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:17.518977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff31c5bf56b643ab8562b0fb5fea2000'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:17.530388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 918/1816] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:16 2022] DELETE /identity/v3/users/ff31c5bf56b643ab8562b0fb5fea2000 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:17.930207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:17.998930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:18.006724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:18.021544 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39b29c-7531-4375-96f0-771eeb842ca0 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:18.088358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:18.089065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:18.089301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:18.089574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] PATH_INFO: `/v3/projects/22a11df5c42a423f85fe10c668825b0d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:18.090700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a11df5c42a423f85fe10c668825b0d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:18.092605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:18.179807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:18.181339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:18.195112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff67c6a-8c83-41b1-83e6-63696e90fbef admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:18.201535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:18.202179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:18.202457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:18.202730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] PATH_INFO: `/v3/projects/7ba8fcb67877424a8348c0cdf7ab62f0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:18.204735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba8fcb67877424a8348c0cdf7ab62f0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:18.206828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:18.251239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a11df5c42a423f85fe10c668825b0d', 'request_id': 'req-94f3b24b-a79e-4631-ae3a-b5959b05a54d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.256292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a11df5c42a423f85fe10c668825b0d', 'request_id': 'req-94f3b24b-a79e-4631-ae3a-b5959b05a54d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.348007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba8fcb67877424a8348c0cdf7ab62f0', 'request_id': 'req-623754f1-ba77-488d-af34-874c6abe7d85'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.354892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba8fcb67877424a8348c0cdf7ab62f0', 'request_id': 'req-623754f1-ba77-488d-af34-874c6abe7d85'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.387239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a11df5c42a423f85fe10c668825b0d', 'request_id': 'req-94f3b24b-a79e-4631-ae3a-b5959b05a54d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.399597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] The token cache is being invalidate because project 22a11df5c42a423f85fe10c668825b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:18.399911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.400599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-94f3b24b-a79e-4631-ae3a-b5959b05a54d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.402229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 899/1817] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:17 2022] DELETE /identity/v3/projects/22a11df5c42a423f85fe10c668825b0d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:18.403415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ce4f7-ff67-408f-919d-ff466f3d3425 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:18.403690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ce4f7-ff67-408f-919d-ff466f3d3425 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:18.403961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ce4f7-ff67-408f-919d-ff466f3d3425 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:18.405695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 900/1818] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:32:18.407095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ce4f7-ff67-408f-919d-ff466f3d3425 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:18.408750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-184ce4f7-ff67-408f-919d-ff466f3d3425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:18.490751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ce4f7-ff67-408f-919d-ff466f3d3425 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:18.495514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:18.496141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:18.496420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:18.496698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] PATH_INFO: `/v3/users/cda1a15139d94b349dbf681c9972348b` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:18.497839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda1a15139d94b349dbf681c9972348b)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:18.499895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:18.505973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba8fcb67877424a8348c0cdf7ab62f0', 'request_id': 'req-623754f1-ba77-488d-af34-874c6abe7d85'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.518183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] The token cache is being invalidate because project 7ba8fcb67877424a8348c0cdf7ab62f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:18.518572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.519310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-623754f1-ba77-488d-af34-874c6abe7d85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.520905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 919/1819] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:18 2022] DELETE /identity/v3/projects/7ba8fcb67877424a8348c0cdf7ab62f0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:32:18.525406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec081a8-680a-4b1a-bb72-a158ece8c2a7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:18.525689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec081a8-680a-4b1a-bb72-a158ece8c2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:18.525965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec081a8-680a-4b1a-bb72-a158ece8c2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:18.545612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-aec081a8-680a-4b1a-bb72-a158ece8c2a7 None None] MFA Rules not processed for user `c7cc35f5ddf0446e90ded5c85f21f37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:18.546951 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aec081a8-680a-4b1a-bb72-a158ece8c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:18.616513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 920/1820] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:32:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:18.617813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49a7d4-694d-436b-bb4c-5560b91f0b4c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:18.618155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49a7d4-694d-436b-bb4c-5560b91f0b4c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:18.618425 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49a7d4-694d-436b-bb4c-5560b91f0b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:18.647057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cd49a7d4-694d-436b-bb4c-5560b91f0b4c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:18.648834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cd49a7d4-694d-436b-bb4c-5560b91f0b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:18.670826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 921/1821] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:18.670826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49a7d4-694d-436b-bb4c-5560b91f0b4c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:18.675600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cd49a7d4-694d-436b-bb4c-5560b91f0b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:18.696020 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49a7d4-694d-436b-bb4c-5560b91f0b4c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:18.702068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45ac71-2520-4b06-8b85-af7d0ab5bd5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:18.702734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45ac71-2520-4b06-8b85-af7d0ab5bd5f service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:18.703084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45ac71-2520-4b06-8b85-af7d0ab5bd5f service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:18.703930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45ac71-2520-4b06-8b85-af7d0ab5bd5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:18.713488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45ac71-2520-4b06-8b85-af7d0ab5bd5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:18.715222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45ac71-2520-4b06-8b85-af7d0ab5bd5f service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:18.726412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 922/1822] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:18.755253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614fa87-adff-42cd-b32b-95b91d986882 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:18.756716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614fa87-adff-42cd-b32b-95b91d986882 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:18.756716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614fa87-adff-42cd-b32b-95b91d986882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:18.770961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda1a15139d94b349dbf681c9972348b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.772455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda1a15139d94b349dbf681c9972348b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.774065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda1a15139d94b349dbf681c9972348b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.793105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda1a15139d94b349dbf681c9972348b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.816770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c614fa87-adff-42cd-b32b-95b91d986882 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:18.818408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda1a15139d94b349dbf681c9972348b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:18.819091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c614fa87-adff-42cd-b32b-95b91d986882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:18.829738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 923/1823] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:18.834234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614fa87-adff-42cd-b32b-95b91d986882 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:18.847302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614fa87-adff-42cd-b32b-95b91d986882 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:18.853625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c1e1-d53c-44bb-beb6-8ca25ac90548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:18.854248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c1e1-d53c-44bb-beb6-8ca25ac90548 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:18.854517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c1e1-d53c-44bb-beb6-8ca25ac90548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:18.854778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c1e1-d53c-44bb-beb6-8ca25ac90548 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:18.855424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360c1e1-d53c-44bb-beb6-8ca25ac90548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:18.855720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360c1e1-d53c-44bb-beb6-8ca25ac90548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:18.871082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360c1e1-d53c-44bb-beb6-8ca25ac90548 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:18.900768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 924/1824] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:32:18.904650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c1e1-d53c-44bb-beb6-8ca25ac90548 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:18.908991 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 901/1825] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:18 2022] DELETE /identity/v3/users/cda1a15139d94b349dbf681c9972348b => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:18.913924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c1e1-d53c-44bb-beb6-8ca25ac90548 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:19.016184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d0762-0f3c-4e46-82df-d90ca782b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:19.016908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d0762-0f3c-4e46-82df-d90ca782b09c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:19.017199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d0762-0f3c-4e46-82df-d90ca782b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:19.017473 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d0762-0f3c-4e46-82df-d90ca782b09c admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:19.018159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d0762-0f3c-4e46-82df-d90ca782b09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:19.021278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d0762-0f3c-4e46-82df-d90ca782b09c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:19.199635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 925/1826] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:18 2022] POST /identity/v3/projects => generated 385 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:32:19.200839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:19.212206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122f66-3e77-4d66-b3db-1b404684997a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:19.310887 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c99eb-2a31-4b0c-af8b-886af6b66f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:19.315146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c99eb-2a31-4b0c-af8b-886af6b66f1f admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:19.315146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c99eb-2a31-4b0c-af8b-886af6b66f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:19.315146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c99eb-2a31-4b0c-af8b-886af6b66f1f admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:19.316425 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c99eb-2a31-4b0c-af8b-886af6b66f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:19.319620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c99eb-2a31-4b0c-af8b-886af6b66f1f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:19.470917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 902/1827] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:19 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:19.472199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d0762-0f3c-4e46-82df-d90ca782b09c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:19.484106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d0762-0f3c-4e46-82df-d90ca782b09c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:19.489946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8d002-d362-4a04-9594-88e814d1b255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:19.490711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8d002-d362-4a04-9594-88e814d1b255 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:19.491207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8d002-d362-4a04-9594-88e814d1b255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:19.491757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8d002-d362-4a04-9594-88e814d1b255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:19.492503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8d002-d362-4a04-9594-88e814d1b255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:19.494648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8d002-d362-4a04-9594-88e814d1b255 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:19.512301 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 926/1828] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:19.517280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c99eb-2a31-4b0c-af8b-886af6b66f1f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:19.528867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c99eb-2a31-4b0c-af8b-886af6b66f1f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:19.539436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982354c-b0a5-4de3-8a82-cb9abab88e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:19.540197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982354c-b0a5-4de3-8a82-cb9abab88e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:19.542340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982354c-b0a5-4de3-8a82-cb9abab88e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:19.542769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982354c-b0a5-4de3-8a82-cb9abab88e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:19.544107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982354c-b0a5-4de3-8a82-cb9abab88e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:19.547643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982354c-b0a5-4de3-8a82-cb9abab88e4c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:19.569477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 903/1829] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:32:19.570901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8d002-d362-4a04-9594-88e814d1b255 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:19.585632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8d002-d362-4a04-9594-88e814d1b255 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:19.595037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb3249-ea36-49d3-98d7-4cf4fedb96b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:19.596397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3249-ea36-49d3-98d7-4cf4fedb96b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:19.596773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3249-ea36-49d3-98d7-4cf4fedb96b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:19.597173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3249-ea36-49d3-98d7-4cf4fedb96b7 admin admin] PATH_INFO: `/v3/projects/7c0f847c4bb74cc69b012a945ab806e1/users/079ca11e098645538fbc708c979c4dca/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:19.629444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb3249-ea36-49d3-98d7-4cf4fedb96b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0f847c4bb74cc69b012a945ab806e1, user_id=079ca11e098645538fbc708c979c4dca, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:19.632612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb3249-ea36-49d3-98d7-4cf4fedb96b7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:19.697425 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44281-046c-44a8-baa1-8a1040e75380 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:19.697710 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44281-046c-44a8-baa1-8a1040e75380 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:19.697986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44281-046c-44a8-baa1-8a1040e75380 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:19.699651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 904/1830] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:32:19.717375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3cb3f-9fa1-4c23-aa56-39026a1e5246 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:19.718115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3cb3f-9fa1-4c23-aa56-39026a1e5246 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:19.718463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3cb3f-9fa1-4c23-aa56-39026a1e5246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:19.746819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-0dc3cb3f-9fa1-4c23-aa56-39026a1e5246 None None] MFA Rules not processed for user `c7cc35f5ddf0446e90ded5c85f21f37d`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:19.748074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0dc3cb3f-9fa1-4c23-aa56-39026a1e5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:19.776310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 927/1831] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:19 2022] PUT /identity/v3/projects/7c0f847c4bb74cc69b012a945ab806e1/users/079ca11e098645538fbc708c979c4dca/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:19.789438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d40d99-b839-471f-b444-74502a785bd4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:19.789798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d40d99-b839-471f-b444-74502a785bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:19.790134 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d40d99-b839-471f-b444-74502a785bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:19.824679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c0d40d99-b839-471f-b444-74502a785bd4 None None] MFA Rules not processed for user `079ca11e098645538fbc708c979c4dca`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:19.826088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c0d40d99-b839-471f-b444-74502a785bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:19.841561 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 905/1832] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:32:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:19.853417 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3cb3f-9fa1-4c23-aa56-39026a1e5246 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:19.855639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0dc3cb3f-9fa1-4c23-aa56-39026a1e5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:19.869728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3cb3f-9fa1-4c23-aa56-39026a1e5246 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:19.903006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 928/1833] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:19.908509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d40d99-b839-471f-b444-74502a785bd4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:19.937478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f0d29-c313-4057-a31b-1f46e2bf4666 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:19.938309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f0d29-c313-4057-a31b-1f46e2bf4666 service placement] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:19.938590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f0d29-c313-4057-a31b-1f46e2bf4666 service placement] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:19.939139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f0d29-c313-4057-a31b-1f46e2bf4666 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:19.948765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f0d29-c313-4057-a31b-1f46e2bf4666 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:19.951253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f0d29-c313-4057-a31b-1f46e2bf4666 service placement] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:19.963782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 906/1834] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:32:19.964784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f0d29-c313-4057-a31b-1f46e2bf4666 service placement] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:19.966247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-262f0d29-c313-4057-a31b-1f46e2bf4666 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:20.012349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d40d99-b839-471f-b444-74502a785bd4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:20.023505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28375aba-abe5-40f1-9dbb-8a77f7082250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:20.024144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28375aba-abe5-40f1-9dbb-8a77f7082250 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:20.024394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28375aba-abe5-40f1-9dbb-8a77f7082250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:20.024745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28375aba-abe5-40f1-9dbb-8a77f7082250 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:20.025411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28375aba-abe5-40f1-9dbb-8a77f7082250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:20.030607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28375aba-abe5-40f1-9dbb-8a77f7082250 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:20.116127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f0d29-c313-4057-a31b-1f46e2bf4666 service placement] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:20.121201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:20.121848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:20.122122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:20.122386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] PATH_INFO: `/v3/projects/1f625c0d6cb842438f8ed9ae72be3826` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:20.123541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f625c0d6cb842438f8ed9ae72be3826)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:20.126278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:20.223675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 929/1835] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:19 2022] POST /identity/v3/projects => generated 383 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:32:20.225180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28375aba-abe5-40f1-9dbb-8a77f7082250 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:20.227442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-28375aba-abe5-40f1-9dbb-8a77f7082250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:20.291260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28375aba-abe5-40f1-9dbb-8a77f7082250 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:20.296346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:20.300631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:20.300631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:20.300631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] PATH_INFO: `/v3/users/a3baf00ac52144bf82e01c0e335a5fd3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:20.300631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3baf00ac52144bf82e01c0e335a5fd3)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:20.301734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:20.408984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f625c0d6cb842438f8ed9ae72be3826', 'request_id': 'req-3062b49d-e7a4-4551-97ff-6c92dbf8933c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.414208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f625c0d6cb842438f8ed9ae72be3826', 'request_id': 'req-3062b49d-e7a4-4551-97ff-6c92dbf8933c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.553005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f625c0d6cb842438f8ed9ae72be3826', 'request_id': 'req-3062b49d-e7a4-4551-97ff-6c92dbf8933c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.564964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] The token cache is being invalidate because project 1f625c0d6cb842438f8ed9ae72be3826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:20.565264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.565910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.567418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 907/1836] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:19 2022] DELETE /identity/v3/projects/1f625c0d6cb842438f8ed9ae72be3826 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:20.568160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:20.569480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:20.612665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3baf00ac52144bf82e01c0e335a5fd3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.640102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062b49d-e7a4-4551-97ff-6c92dbf8933c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:20.646141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66da2a-3f4c-435a-a870-238b84c672c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:20.650391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66da2a-3f4c-435a-a870-238b84c672c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:20.650391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66da2a-3f4c-435a-a870-238b84c672c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:20.650391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66da2a-3f4c-435a-a870-238b84c672c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:20.650391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66da2a-3f4c-435a-a870-238b84c672c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:20.650391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66da2a-3f4c-435a-a870-238b84c672c4 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:20.753862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3baf00ac52144bf82e01c0e335a5fd3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.764415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 908/1837] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:20 2022] POST /identity/v3/users => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:20.766668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc9272-b3d2-4f95-9424-8b7a1817d593 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:20.768179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc9272-b3d2-4f95-9424-8b7a1817d593 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:20.768481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc9272-b3d2-4f95-9424-8b7a1817d593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:20.778495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3baf00ac52144bf82e01c0e335a5fd3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.779723 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3baf00ac52144bf82e01c0e335a5fd3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.803860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3baf00ac52144bf82e01c0e335a5fd3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:20.807662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-2efc9272-b3d2-4f95-9424-8b7a1817d593 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:20.810625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2efc9272-b3d2-4f95-9424-8b7a1817d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:20.817687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 930/1838] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:20 2022] DELETE /identity/v3/users/a3baf00ac52144bf82e01c0e335a5fd3 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:20.818453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:20.829540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:20.835408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 909/1839] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:20.839772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:20.843402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479259f-068e-473b-a906-ab26395e290a None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:20.847113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479259f-068e-473b-a906-ab26395e290a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:20.847113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479259f-068e-473b-a906-ab26395e290a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:20.866179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e479259f-068e-473b-a906-ab26395e290a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:20.866557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e479259f-068e-473b-a906-ab26395e290a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:20.873627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:20.936561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83e971-2a5a-425c-a93c-c018c78c50d4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:20.943825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bb2f8-63c4-44e7-869c-ad92d72cdaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:20.944532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bb2f8-63c4-44e7-869c-ad92d72cdaee admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:20.944828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bb2f8-63c4-44e7-869c-ad92d72cdaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:20.945350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bb2f8-63c4-44e7-869c-ad92d72cdaee admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:20.946726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bb2f8-63c4-44e7-869c-ad92d72cdaee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:20.948389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bb2f8-63c4-44e7-869c-ad92d72cdaee admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:20.967347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 931/1840] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:20.969234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bb2f8-63c4-44e7-869c-ad92d72cdaee admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:20.981901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bb2f8-63c4-44e7-869c-ad92d72cdaee admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:20.988851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af12e6-ea9b-46d5-80b3-e33fbf829dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:20.989633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af12e6-ea9b-46d5-80b3-e33fbf829dae admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:20.989902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af12e6-ea9b-46d5-80b3-e33fbf829dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:20.990103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af12e6-ea9b-46d5-80b3-e33fbf829dae admin admin] PATH_INFO: `/v3/projects/49f27d7caa1f4c01bbf20020baac5ba3/users/0107f87945fd43d18862b8136f90279f/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:20.992538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 910/1841] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:21.006345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479259f-068e-473b-a906-ab26395e290a None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.025705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479259f-068e-473b-a906-ab26395e290a None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.030450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5da630-f949-41b4-a328-d920d942ff39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.031221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da630-f949-41b4-a328-d920d942ff39 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.031694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da630-f949-41b4-a328-d920d942ff39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.032055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da630-f949-41b4-a328-d920d942ff39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.032615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af12e6-ea9b-46d5-80b3-e33fbf829dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f27d7caa1f4c01bbf20020baac5ba3, user_id=0107f87945fd43d18862b8136f90279f, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.032915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5da630-f949-41b4-a328-d920d942ff39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:21.033288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5da630-f949-41b4-a328-d920d942ff39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.034948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5da630-f949-41b4-a328-d920d942ff39 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.035319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af12e6-ea9b-46d5-80b3-e33fbf829dae admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.051265 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 911/1842] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:32:21.052283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5da630-f949-41b4-a328-d920d942ff39 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.062099 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5da630-f949-41b4-a328-d920d942ff39 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.067653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddc235-4f7d-4116-b471-9efe5ce793d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.068245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc235-4f7d-4116-b471-9efe5ce793d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.068520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc235-4f7d-4116-b471-9efe5ce793d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.068781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc235-4f7d-4116-b471-9efe5ce793d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.069416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddc235-4f7d-4116-b471-9efe5ce793d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.071565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddc235-4f7d-4116-b471-9efe5ce793d4 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.113839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 932/1843] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:20 2022] PUT /identity/v3/projects/49f27d7caa1f4c01bbf20020baac5ba3/users/0107f87945fd43d18862b8136f90279f/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:21.118011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af12e6-ea9b-46d5-80b3-e33fbf829dae admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.186334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af12e6-ea9b-46d5-80b3-e33fbf829dae admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.191529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7d135-d4df-412c-b9ab-84b37c5fbb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.192130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7d135-d4df-412c-b9ab-84b37c5fbb5c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.192405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7d135-d4df-412c-b9ab-84b37c5fbb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.192672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7d135-d4df-412c-b9ab-84b37c5fbb5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.193470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7d135-d4df-412c-b9ab-84b37c5fbb5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.195052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7d135-d4df-412c-b9ab-84b37c5fbb5c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.205540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 933/1844] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:21.206340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7d135-d4df-412c-b9ab-84b37c5fbb5c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.208100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d9c7d135-d4df-412c-b9ab-84b37c5fbb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:21.222661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7d135-d4df-412c-b9ab-84b37c5fbb5c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.227621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.228316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.228864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.229136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] PATH_INFO: `/v3/projects/de2dd5c0338541d68355d679b053bbf0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.230617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 912/1845] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:21 2022] POST /identity/v3/projects => generated 389 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:21.230720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2dd5c0338541d68355d679b053bbf0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.231065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddc235-4f7d-4116-b471-9efe5ce793d4 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.232664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.320892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddc235-4f7d-4116-b471-9efe5ce793d4 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.328187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7879d11-f591-4d51-bb74-f221451cf89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.328510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879d11-f591-4d51-bb74-f221451cf89b admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.328970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879d11-f591-4d51-bb74-f221451cf89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.329503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879d11-f591-4d51-bb74-f221451cf89b admin admin] PATH_INFO: `/v3/projects/49f27d7caa1f4c01bbf20020baac5ba3/users/0107f87945fd43d18862b8136f90279f/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.331863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7879d11-f591-4d51-bb74-f221451cf89b admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f27d7caa1f4c01bbf20020baac5ba3, user_id=0107f87945fd43d18862b8136f90279f, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.334617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7879d11-f591-4d51-bb74-f221451cf89b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.346979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 913/1846] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:21 2022] PUT /identity/v3/projects/49f27d7caa1f4c01bbf20020baac5ba3/users/0107f87945fd43d18862b8136f90279f/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:21.347856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7879d11-f591-4d51-bb74-f221451cf89b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.362426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2dd5c0338541d68355d679b053bbf0', 'request_id': 'req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:21.363361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:21.370188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2dd5c0338541d68355d679b053bbf0', 'request_id': 'req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:21.455262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7879d11-f591-4d51-bb74-f221451cf89b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.507166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2dd5c0338541d68355d679b053bbf0', 'request_id': 'req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:21.516896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe00bb-e322-40d3-bb01-2fe6f41efa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.517539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe00bb-e322-40d3-bb01-2fe6f41efa6b admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.517815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe00bb-e322-40d3-bb01-2fe6f41efa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.518084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe00bb-e322-40d3-bb01-2fe6f41efa6b admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.518705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe00bb-e322-40d3-bb01-2fe6f41efa6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.519191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] The token cache is being invalidate because project de2dd5c0338541d68355d679b053bbf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:21.519612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:21.520421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe7b93c-ef7d-43a7-83c2-a956a4ca9b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:21.520662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe00bb-e322-40d3-bb01-2fe6f41efa6b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.522167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 934/1847] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:21 2022] DELETE /identity/v3/projects/de2dd5c0338541d68355d679b053bbf0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:21.523335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44d9cd-db2a-4a6d-93a8-2a73281938a3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.523613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44d9cd-db2a-4a6d-93a8-2a73281938a3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.523880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44d9cd-db2a-4a6d-93a8-2a73281938a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.553921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5d44d9cd-db2a-4a6d-93a8-2a73281938a3 None None] MFA Rules not processed for user `0107f87945fd43d18862b8136f90279f`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:21.555042 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5d44d9cd-db2a-4a6d-93a8-2a73281938a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:21.632072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 914/1848] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:21 2022] POST /identity/v3/users => generated 295 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:21.632072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe00bb-e322-40d3-bb01-2fe6f41efa6b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.632072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-aebe00bb-e322-40d3-bb01-2fe6f41efa6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:21.639607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 935/1849] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:21.639607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44d9cd-db2a-4a6d-93a8-2a73281938a3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.639607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5d44d9cd-db2a-4a6d-93a8-2a73281938a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:21.642858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe00bb-e322-40d3-bb01-2fe6f41efa6b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.649848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.650920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.651201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.651475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] PATH_INFO: `/v3/users/eb6dfc33a8d348fab9335840a9d67bd1` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.652737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb6dfc33a8d348fab9335840a9d67bd1)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.655281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.660047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44d9cd-db2a-4a6d-93a8-2a73281938a3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.666536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d321b9-f078-4b4f-b81f-b9bbc08bf777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.667241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d321b9-f078-4b4f-b81f-b9bbc08bf777 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.667542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d321b9-f078-4b4f-b81f-b9bbc08bf777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.667886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d321b9-f078-4b4f-b81f-b9bbc08bf777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.668444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d321b9-f078-4b4f-b81f-b9bbc08bf777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.670206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d321b9-f078-4b4f-b81f-b9bbc08bf777 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.681371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 936/1850] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:21.682245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d321b9-f078-4b4f-b81f-b9bbc08bf777 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.683966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c0d321b9-f078-4b4f-b81f-b9bbc08bf777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:21.742551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d321b9-f078-4b4f-b81f-b9bbc08bf777 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.748351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156226b4-f6d5-4a3a-b39b-4774426b6cd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.749132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156226b4-f6d5-4a3a-b39b-4774426b6cd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.749440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156226b4-f6d5-4a3a-b39b-4774426b6cd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.749709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156226b4-f6d5-4a3a-b39b-4774426b6cd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.751571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-156226b4-f6d5-4a3a-b39b-4774426b6cd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:21.818597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156226b4-f6d5-4a3a-b39b-4774426b6cd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.820050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156226b4-f6d5-4a3a-b39b-4774426b6cd2 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.830094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 937/1851] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:21.830953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156226b4-f6d5-4a3a-b39b-4774426b6cd2 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.855359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156226b4-f6d5-4a3a-b39b-4774426b6cd2 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.865495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e8048-328a-48e3-b68d-a5ddfcd77066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.866405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e8048-328a-48e3-b68d-a5ddfcd77066 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.866910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e8048-328a-48e3-b68d-a5ddfcd77066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.867189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e8048-328a-48e3-b68d-a5ddfcd77066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.868527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e8048-328a-48e3-b68d-a5ddfcd77066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.873700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e8048-328a-48e3-b68d-a5ddfcd77066 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.894556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 938/1852] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:21.895300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e8048-328a-48e3-b68d-a5ddfcd77066 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:21.912415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e8048-328a-48e3-b68d-a5ddfcd77066 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:21.921659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c31a13-ac7e-4975-a33c-c932d2579225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:21.923181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c31a13-ac7e-4975-a33c-c932d2579225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:21.923444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c31a13-ac7e-4975-a33c-c932d2579225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:21.923724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c31a13-ac7e-4975-a33c-c932d2579225 admin admin] PATH_INFO: `/v3/projects/407577115c3b4744986047c07520f905/users/db6f0504d59a498d8668ae736fec73b6/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:21.943724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c31a13-ac7e-4975-a33c-c932d2579225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=407577115c3b4744986047c07520f905, user_id=db6f0504d59a498d8668ae736fec73b6, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:21.947459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb6dfc33a8d348fab9335840a9d67bd1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:21.948824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb6dfc33a8d348fab9335840a9d67bd1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:21.949629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb6dfc33a8d348fab9335840a9d67bd1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:21.958491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb6dfc33a8d348fab9335840a9d67bd1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:21.959303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:21.962087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c31a13-ac7e-4975-a33c-c932d2579225 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:21.975631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb6dfc33a8d348fab9335840a9d67bd1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:22.093587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 939/1853] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:21 2022] PUT /identity/v3/projects/407577115c3b4744986047c07520f905/users/db6f0504d59a498d8668ae736fec73b6/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:32:22.094825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 915/1854] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:21 2022] DELETE /identity/v3/users/eb6dfc33a8d348fab9335840a9d67bd1 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:22.095813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d095e-bffe-465a-9f6b-ef7350e542c9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:22.096093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d095e-bffe-465a-9f6b-ef7350e542c9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:22.096362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d095e-bffe-465a-9f6b-ef7350e542c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:22.147645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4f0d095e-bffe-465a-9f6b-ef7350e542c9 None None] MFA Rules not processed for user `db6f0504d59a498d8668ae736fec73b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:22.148936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4f0d095e-bffe-465a-9f6b-ef7350e542c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:22.239930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 940/1855] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:22.245016 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:22.247417 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:22.341115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35529f94-9551-4ed3-9cf5-a68d0fcda4af admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:22.353081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf83ad-6e31-4336-bb65-660135929679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:22.353081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf83ad-6e31-4336-bb65-660135929679 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:22.353081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf83ad-6e31-4336-bb65-660135929679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:22.353081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf83ad-6e31-4336-bb65-660135929679 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:22.353375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf83ad-6e31-4336-bb65-660135929679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:22.353375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf83ad-6e31-4336-bb65-660135929679 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:22.404297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d764b-c43c-46cf-9b4f-814c4b7dc35d None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:22.404614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d764b-c43c-46cf-9b4f-814c4b7dc35d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:22.404945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d764b-c43c-46cf-9b4f-814c4b7dc35d None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:22.406613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 941/1856] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:32:22.415231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694c6e-b55b-4760-9646-8a453cb9e2b3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:22.415641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694c6e-b55b-4760-9646-8a453cb9e2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:22.416105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694c6e-b55b-4760-9646-8a453cb9e2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:22.436104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ae694c6e-b55b-4760-9646-8a453cb9e2b3 None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:22.437872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ae694c6e-b55b-4760-9646-8a453cb9e2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:22.498302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:22.520611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 916/1857] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:22 2022] POST /identity/v3/projects => generated 387 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:22.523639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf83ad-6e31-4336-bb65-660135929679 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:22.526051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-bbcf83ad-6e31-4336-bb65-660135929679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:22.570416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 942/1858] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:32:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:22.571307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694c6e-b55b-4760-9646-8a453cb9e2b3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:22.592721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694c6e-b55b-4760-9646-8a453cb9e2b3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:22.699920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf83ad-6e31-4336-bb65-660135929679 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:22.706166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:22.707067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:22.707445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:22.707553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c287989-3297-4441-8fb2-028a3e77c754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:22.707787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] PATH_INFO: `/v3/projects/36c8c75d1f364fe3aceb9f0577514756` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:22.708256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c287989-3297-4441-8fb2-028a3e77c754 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:22.708657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c287989-3297-4441-8fb2-028a3e77c754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:22.708976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c287989-3297-4441-8fb2-028a3e77c754 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:22.709614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c287989-3297-4441-8fb2-028a3e77c754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:22.709803 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] RBAC: Authorizing `identity:delete_project(project_id=36c8c75d1f364fe3aceb9f0577514756)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:22.711563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c287989-3297-4441-8fb2-028a3e77c754 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:22.711834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:22.844166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 943/1859] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:22 2022] POST /identity/v3/users => generated 300 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:22.844166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c287989-3297-4441-8fb2-028a3e77c754 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:22.846047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1c287989-3297-4441-8fb2-028a3e77c754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:22.865069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36c8c75d1f364fe3aceb9f0577514756', 'request_id': 'req-79cede46-1e9a-4679-8b55-9dfe80cca38b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:22.872557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36c8c75d1f364fe3aceb9f0577514756', 'request_id': 'req-79cede46-1e9a-4679-8b55-9dfe80cca38b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:22.935706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c287989-3297-4441-8fb2-028a3e77c754 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:22.987186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36c8c75d1f364fe3aceb9f0577514756', 'request_id': 'req-79cede46-1e9a-4679-8b55-9dfe80cca38b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:23.003665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] The token cache is being invalidate because project 36c8c75d1f364fe3aceb9f0577514756 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:23.004889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:23.006682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:23.020404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed7b3c-d5a8-4628-a162-591da3643e52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.021073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed7b3c-d5a8-4628-a162-591da3643e52 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.021349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed7b3c-d5a8-4628-a162-591da3643e52 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.021661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed7b3c-d5a8-4628-a162-591da3643e52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.024053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f4ed7b3c-d5a8-4628-a162-591da3643e52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:23.025650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 917/1860] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:22 2022] DELETE /identity/v3/projects/36c8c75d1f364fe3aceb9f0577514756 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:23.027148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.030231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:23.041566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed7b3c-d5a8-4628-a162-591da3643e52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.043326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed7b3c-d5a8-4628-a162-591da3643e52 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:23.055896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 944/1861] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:23.070428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f9345-53d6-4b8e-8f87-4a66bb78fa1c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.070837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f9345-53d6-4b8e-8f87-4a66bb78fa1c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.071214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f9345-53d6-4b8e-8f87-4a66bb78fa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.104920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4c9f9345-53d6-4b8e-8f87-4a66bb78fa1c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:23.107687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4c9f9345-53d6-4b8e-8f87-4a66bb78fa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:23.143292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cede46-1e9a-4679-8b55-9dfe80cca38b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:23.148534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:23.155748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92681e4a-2b2b-4b6a-8162-7d8335367c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.157631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92681e4a-2b2b-4b6a-8162-7d8335367c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.158440 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92681e4a-2b2b-4b6a-8162-7d8335367c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.158857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92681e4a-2b2b-4b6a-8162-7d8335367c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.159564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92681e4a-2b2b-4b6a-8162-7d8335367c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.161197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92681e4a-2b2b-4b6a-8162-7d8335367c3b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:23.182651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 918/1862] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:23.183882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92681e4a-2b2b-4b6a-8162-7d8335367c3b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.195529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92681e4a-2b2b-4b6a-8162-7d8335367c3b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:23.200509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072aedb-4e0b-4373-b3fa-2654dd416e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.201203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072aedb-4e0b-4373-b3fa-2654dd416e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.207828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072aedb-4e0b-4373-b3fa-2654dd416e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.207828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072aedb-4e0b-4373-b3fa-2654dd416e47 admin admin] PATH_INFO: `/v3/projects/9cdd0a766f364a549ec659fab593d5cd/users/118a9620664247cbab3a24dd4cf3bdf1/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.210211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 945/1863] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:23.218137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072aedb-4e0b-4373-b3fa-2654dd416e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cdd0a766f364a549ec659fab593d5cd, user_id=118a9620664247cbab3a24dd4cf3bdf1, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.219488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020177-bd7b-42b0-9600-7857116b06f7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.220064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020177-bd7b-42b0-9600-7857116b06f7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.220064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9020177-bd7b-42b0-9600-7857116b06f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.221752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072aedb-4e0b-4373-b3fa-2654dd416e47 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:23.245611 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a9020177-bd7b-42b0-9600-7857116b06f7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:23.246877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a9020177-bd7b-42b0-9600-7857116b06f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:23.255841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 946/1864] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:23.260206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9020177-bd7b-42b0-9600-7857116b06f7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.270140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9020177-bd7b-42b0-9600-7857116b06f7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:23.279298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a834c8-3beb-4aef-a5d3-83d0840e9e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.279471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a834c8-3beb-4aef-a5d3-83d0840e9e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.282144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a834c8-3beb-4aef-a5d3-83d0840e9e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.282144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a834c8-3beb-4aef-a5d3-83d0840e9e64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.283110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a834c8-3beb-4aef-a5d3-83d0840e9e64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:23.283624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a834c8-3beb-4aef-a5d3-83d0840e9e64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.286735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a834c8-3beb-4aef-a5d3-83d0840e9e64 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:23.304498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 947/1865] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:32:23.316057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a834c8-3beb-4aef-a5d3-83d0840e9e64 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.325096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a834c8-3beb-4aef-a5d3-83d0840e9e64 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:23.330492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69078a-e6c1-49e2-8057-82130fe4b707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.331868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69078a-e6c1-49e2-8057-82130fe4b707 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.332211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69078a-e6c1-49e2-8057-82130fe4b707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.332517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69078a-e6c1-49e2-8057-82130fe4b707 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.333300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69078a-e6c1-49e2-8057-82130fe4b707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.335334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69078a-e6c1-49e2-8057-82130fe4b707 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:23.357513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 919/1866] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:23 2022] PUT /identity/v3/projects/9cdd0a766f364a549ec659fab593d5cd/users/118a9620664247cbab3a24dd4cf3bdf1/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:23.365534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072aedb-4e0b-4373-b3fa-2654dd416e47 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.470424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 948/1867] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:23 2022] POST /identity/v3/projects => generated 407 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:32:23.474897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69078a-e6c1-49e2-8057-82130fe4b707 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.498276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072aedb-4e0b-4373-b3fa-2654dd416e47 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:23.617884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be089a-e86a-454c-9b01-e640ebd70ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.620441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be089a-e86a-454c-9b01-e640ebd70ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.620835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69078a-e6c1-49e2-8057-82130fe4b707 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:23.621286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be089a-e86a-454c-9b01-e640ebd70ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.622295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be089a-e86a-454c-9b01-e640ebd70ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.623340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be089a-e86a-454c-9b01-e640ebd70ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.626566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603be8b-3ba8-4b9b-b85f-0a51005531a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.629936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be089a-e86a-454c-9b01-e640ebd70ac5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:23.630081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603be8b-3ba8-4b9b-b85f-0a51005531a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.630081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603be8b-3ba8-4b9b-b85f-0a51005531a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.630081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603be8b-3ba8-4b9b-b85f-0a51005531a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.630081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603be8b-3ba8-4b9b-b85f-0a51005531a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.631747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603be8b-3ba8-4b9b-b85f-0a51005531a9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:23.644258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 920/1868] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:23.657145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be089a-e86a-454c-9b01-e640ebd70ac5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.673582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be089a-e86a-454c-9b01-e640ebd70ac5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:23.678796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe3dfc-6b9f-4c95-8092-dc511810168c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.679488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe3dfc-6b9f-4c95-8092-dc511810168c admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.679755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe3dfc-6b9f-4c95-8092-dc511810168c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.680040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe3dfc-6b9f-4c95-8092-dc511810168c admin admin] PATH_INFO: `/v3/projects/9cdd0a766f364a549ec659fab593d5cd/users/118a9620664247cbab3a24dd4cf3bdf1/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.682853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe3dfc-6b9f-4c95-8092-dc511810168c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cdd0a766f364a549ec659fab593d5cd, user_id=118a9620664247cbab3a24dd4cf3bdf1, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.685933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe3dfc-6b9f-4c95-8092-dc511810168c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:23.700525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 921/1869] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:23 2022] PUT /identity/v3/projects/9cdd0a766f364a549ec659fab593d5cd/users/118a9620664247cbab3a24dd4cf3bdf1/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:23.712625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bcb90-927a-42b4-adee-90637b0e0f93 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.712952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bcb90-927a-42b4-adee-90637b0e0f93 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.713238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bcb90-927a-42b4-adee-90637b0e0f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.753176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 949/1870] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:23 2022] POST /identity/v3/users => generated 304 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:23.754361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603be8b-3ba8-4b9b-b85f-0a51005531a9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.765913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ca3bcb90-927a-42b4-adee-90637b0e0f93 None None] MFA Rules not processed for user `118a9620664247cbab3a24dd4cf3bdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:23.769353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ca3bcb90-927a-42b4-adee-90637b0e0f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:23.770538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603be8b-3ba8-4b9b-b85f-0a51005531a9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:23.846044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc527e2-3911-4ad0-89d6-8c0c41d11754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.846712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc527e2-3911-4ad0-89d6-8c0c41d11754 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.846949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc527e2-3911-4ad0-89d6-8c0c41d11754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.847220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc527e2-3911-4ad0-89d6-8c0c41d11754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.848012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc527e2-3911-4ad0-89d6-8c0c41d11754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.850781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc527e2-3911-4ad0-89d6-8c0c41d11754 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:23.877618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 950/1871] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:23.899384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 922/1872] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:23.899861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc527e2-3911-4ad0-89d6-8c0c41d11754 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.912571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bcb90-927a-42b4-adee-90637b0e0f93 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:23.914696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ca3bcb90-927a-42b4-adee-90637b0e0f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:23.941814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc527e2-3911-4ad0-89d6-8c0c41d11754 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:23.960283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa03cf4-90ee-4cdb-a3e5-e925248e0f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:23.963536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa03cf4-90ee-4cdb-a3e5-e925248e0f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:23.963857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa03cf4-90ee-4cdb-a3e5-e925248e0f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:23.965197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa03cf4-90ee-4cdb-a3e5-e925248e0f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:23.970761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa03cf4-90ee-4cdb-a3e5-e925248e0f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:23.970761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa03cf4-90ee-4cdb-a3e5-e925248e0f9c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:24.003432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 951/1873] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:24.012412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa03cf4-90ee-4cdb-a3e5-e925248e0f9c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:24.027367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bcb90-927a-42b4-adee-90637b0e0f93 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:24.032973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670179f-dbf7-4690-87e5-d610c7f12f59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:24.033853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670179f-dbf7-4690-87e5-d610c7f12f59 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:24.034199 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670179f-dbf7-4690-87e5-d610c7f12f59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:24.034474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670179f-dbf7-4690-87e5-d610c7f12f59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:24.036035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa03cf4-90ee-4cdb-a3e5-e925248e0f9c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:24.042028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d460b16-4729-4d2b-99de-5b51afca6f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:24.043714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d460b16-4729-4d2b-99de-5b51afca6f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:24.044306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d460b16-4729-4d2b-99de-5b51afca6f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:24.044869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d460b16-4729-4d2b-99de-5b51afca6f2f admin admin] PATH_INFO: `/v3/projects/34e391f0088d4ff7b85efa0419e3057e/users/b87a5fc6155a41fa8d7cd61762d24278/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:24.045400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670179f-dbf7-4690-87e5-d610c7f12f59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:24.047132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670179f-dbf7-4690-87e5-d610c7f12f59 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:24.064138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 923/1874] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:32:24.069171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d460b16-4729-4d2b-99de-5b51afca6f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e391f0088d4ff7b85efa0419e3057e, user_id=b87a5fc6155a41fa8d7cd61762d24278, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:24.074536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d460b16-4729-4d2b-99de-5b51afca6f2f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:24.204827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 952/1875] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:24 2022] PUT /identity/v3/projects/34e391f0088d4ff7b85efa0419e3057e/users/b87a5fc6155a41fa8d7cd61762d24278/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:24.217646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b10b38-8fd3-4b09-9d9f-cfc233177f74 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:24.217840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b10b38-8fd3-4b09-9d9f-cfc233177f74 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:24.218099 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b10b38-8fd3-4b09-9d9f-cfc233177f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:24.261527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e7b10b38-8fd3-4b09-9d9f-cfc233177f74 None None] MFA Rules not processed for user `b87a5fc6155a41fa8d7cd61762d24278`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:24.263818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e7b10b38-8fd3-4b09-9d9f-cfc233177f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:24.298584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d460b16-4729-4d2b-99de-5b51afca6f2f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:24.368205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 924/1876] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:24 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:24.369827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b10b38-8fd3-4b09-9d9f-cfc233177f74 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:24.382893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d460b16-4729-4d2b-99de-5b51afca6f2f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:24.385384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b10b38-8fd3-4b09-9d9f-cfc233177f74 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:24.390342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:24.391163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:24.391538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:24.391783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] PATH_INFO: `/v3/users/db6f0504d59a498d8668ae736fec73b6` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:24.393055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e1520-0fbc-4f44-80e7-e5f78b815c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:24.393204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6f0504d59a498d8668ae736fec73b6)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:24.394071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e1520-0fbc-4f44-80e7-e5f78b815c77 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:24.394543 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e1520-0fbc-4f44-80e7-e5f78b815c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:24.395578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:24.395857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e1520-0fbc-4f44-80e7-e5f78b815c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:24.396716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e1520-0fbc-4f44-80e7-e5f78b815c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:24.398852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e1520-0fbc-4f44-80e7-e5f78b815c77 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:24.445204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:24.544378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 925/1877] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:24 2022] POST /identity/v3/projects => generated 407 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:24.551087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e1520-0fbc-4f44-80e7-e5f78b815c77 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:24.662278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e1520-0fbc-4f44-80e7-e5f78b815c77 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:24.676062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c233b6-83d5-4774-b491-6c3fba5e5eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:24.677255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c233b6-83d5-4774-b491-6c3fba5e5eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:24.677783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c233b6-83d5-4774-b491-6c3fba5e5eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:24.678494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c233b6-83d5-4774-b491-6c3fba5e5eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:24.679609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c233b6-83d5-4774-b491-6c3fba5e5eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:24.682635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c233b6-83d5-4774-b491-6c3fba5e5eb1 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:24.698185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6f0504d59a498d8668ae736fec73b6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:24.827242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6f0504d59a498d8668ae736fec73b6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:24.848291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6f0504d59a498d8668ae736fec73b6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:24.849392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6f0504d59a498d8668ae736fec73b6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:24.857827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b0446b9f-1d3a-4eab-b9cc-082bc0a61bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6f0504d59a498d8668ae736fec73b6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:24.871354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 926/1878] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:24 2022] POST /identity/v3/users => generated 310 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:24.872327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c233b6-83d5-4774-b491-6c3fba5e5eb1 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:24.876691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-25c233b6-83d5-4774-b491-6c3fba5e5eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:24.884102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 953/1879] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:24 2022] DELETE /identity/v3/users/db6f0504d59a498d8668ae736fec73b6 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:24.888769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d5566-0f40-4b54-b9bf-b380d0331fc4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:24.889082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d5566-0f40-4b54-b9bf-b380d0331fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:24.889389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d5566-0f40-4b54-b9bf-b380d0331fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:24.902574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c233b6-83d5-4774-b491-6c3fba5e5eb1 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:24.926265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-796d5566-0f40-4b54-b9bf-b380d0331fc4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:24.927516 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-796d5566-0f40-4b54-b9bf-b380d0331fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:24.982363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18df024-bfcf-4d70-9c71-7a2a3e0be3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:24.983007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18df024-bfcf-4d70-9c71-7a2a3e0be3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:24.983331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18df024-bfcf-4d70-9c71-7a2a3e0be3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:24.984373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18df024-bfcf-4d70-9c71-7a2a3e0be3a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:24.985332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18df024-bfcf-4d70-9c71-7a2a3e0be3a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:24.987623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18df024-bfcf-4d70-9c71-7a2a3e0be3a6 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:24.998272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 927/1880] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:25.003181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18df024-bfcf-4d70-9c71-7a2a3e0be3a6 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:25.023486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 954/1881] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:25.057146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18df024-bfcf-4d70-9c71-7a2a3e0be3a6 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:25.067810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d5566-0f40-4b54-b9bf-b380d0331fc4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:25.067810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-796d5566-0f40-4b54-b9bf-b380d0331fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:25.072990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4546387-dee5-45a7-9ab0-8557ad5c5466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:25.074611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4546387-dee5-45a7-9ab0-8557ad5c5466 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:25.074895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4546387-dee5-45a7-9ab0-8557ad5c5466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:25.075170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4546387-dee5-45a7-9ab0-8557ad5c5466 admin admin] PATH_INFO: `/v3/projects/e929fdf7aedf429fa61ed39cdb5f1991/users/cd13d8419d3d44c1ad371979d722406b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:25.103166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4546387-dee5-45a7-9ab0-8557ad5c5466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e929fdf7aedf429fa61ed39cdb5f1991, user_id=cd13d8419d3d44c1ad371979d722406b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:25.107309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4546387-dee5-45a7-9ab0-8557ad5c5466 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:25.226457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d5566-0f40-4b54-b9bf-b380d0331fc4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:25.237375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 928/1882] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:25 2022] PUT /identity/v3/projects/e929fdf7aedf429fa61ed39cdb5f1991/users/cd13d8419d3d44c1ad371979d722406b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:32:25.238781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4546387-dee5-45a7-9ab0-8557ad5c5466 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:25.353579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e916180-dc11-4591-9538-898e13a21268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:25.354203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e916180-dc11-4591-9538-898e13a21268 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:25.354730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e916180-dc11-4591-9538-898e13a21268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:25.355110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e916180-dc11-4591-9538-898e13a21268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:25.365542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4546387-dee5-45a7-9ab0-8557ad5c5466 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:25.366442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e916180-dc11-4591-9538-898e13a21268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:25.367997 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e916180-dc11-4591-9538-898e13a21268 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:25.370702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0524a8-9525-4e18-82ad-e0f74307d15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:25.371313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0524a8-9525-4e18-82ad-e0f74307d15a admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:25.371591 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0524a8-9525-4e18-82ad-e0f74307d15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:25.371866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0524a8-9525-4e18-82ad-e0f74307d15a admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:25.372501 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0524a8-9525-4e18-82ad-e0f74307d15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:25.374157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0524a8-9525-4e18-82ad-e0f74307d15a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:25.379173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 955/1883] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:32:25.395279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 929/1884] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:25.396402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e916180-dc11-4591-9538-898e13a21268 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:25.407780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e916180-dc11-4591-9538-898e13a21268 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:25.413196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b2cb3-27c7-40c8-a716-3855187e8740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:25.414070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b2cb3-27c7-40c8-a716-3855187e8740 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:25.414197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b2cb3-27c7-40c8-a716-3855187e8740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:25.414398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b2cb3-27c7-40c8-a716-3855187e8740 admin admin] PATH_INFO: `/v3/projects/e929fdf7aedf429fa61ed39cdb5f1991/users/cd13d8419d3d44c1ad371979d722406b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:25.416892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b2cb3-27c7-40c8-a716-3855187e8740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e929fdf7aedf429fa61ed39cdb5f1991, user_id=cd13d8419d3d44c1ad371979d722406b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:25.420971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b2cb3-27c7-40c8-a716-3855187e8740 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:25.436833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 956/1885] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:25 2022] PUT /identity/v3/projects/e929fdf7aedf429fa61ed39cdb5f1991/users/cd13d8419d3d44c1ad371979d722406b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:32:25.451908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bde98-4ff8-483d-87d6-7461b7efc442 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:25.452194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bde98-4ff8-483d-87d6-7461b7efc442 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:25.452579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bde98-4ff8-483d-87d6-7461b7efc442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:25.495509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-229bde98-4ff8-483d-87d6-7461b7efc442 None None] MFA Rules not processed for user `cd13d8419d3d44c1ad371979d722406b`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:25.496872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-229bde98-4ff8-483d-87d6-7461b7efc442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:25.571303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 930/1886] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:25 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:25.582899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b2cb3-27c7-40c8-a716-3855187e8740 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:25.584616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-078b2cb3-27c7-40c8-a716-3855187e8740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:25.694931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b2cb3-27c7-40c8-a716-3855187e8740 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:25.699049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7d18f-be47-4c6b-a70d-132eaed447ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:25.699664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7d18f-be47-4c6b-a70d-132eaed447ff service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:25.699941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7d18f-be47-4c6b-a70d-132eaed447ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:25.700651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7d18f-be47-4c6b-a70d-132eaed447ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:25.712031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7d18f-be47-4c6b-a70d-132eaed447ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:25.713489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7d18f-be47-4c6b-a70d-132eaed447ff service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:25.725751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 957/1887] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:25 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:26.092338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bde98-4ff8-483d-87d6-7461b7efc442 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:26.092338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-229bde98-4ff8-483d-87d6-7461b7efc442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:26.249013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bde98-4ff8-483d-87d6-7461b7efc442 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:26.260008 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:26.260667 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:26.260988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:26.261452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] PATH_INFO: `/v3/projects/407577115c3b4744986047c07520f905` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:26.262695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=407577115c3b4744986047c07520f905)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:26.266638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:26.369258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7d18f-be47-4c6b-a70d-132eaed447ff service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:26.370938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-19a7d18f-be47-4c6b-a70d-132eaed447ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:26.385953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7d18f-be47-4c6b-a70d-132eaed447ff service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:26.390664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776bdde-96ab-4a9c-98ca-c4b72040d3e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:26.391399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776bdde-96ab-4a9c-98ca-c4b72040d3e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:26.391800 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776bdde-96ab-4a9c-98ca-c4b72040d3e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:26.392077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776bdde-96ab-4a9c-98ca-c4b72040d3e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:26.393914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4776bdde-96ab-4a9c-98ca-c4b72040d3e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:26.457606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407577115c3b4744986047c07520f905', 'request_id': 'req-2c80f496-4b41-4019-82c4-ad1602129e8e'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:26.463392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407577115c3b4744986047c07520f905', 'request_id': 'req-2c80f496-4b41-4019-82c4-ad1602129e8e'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:26.464781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776bdde-96ab-4a9c-98ca-c4b72040d3e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:26.465976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776bdde-96ab-4a9c-98ca-c4b72040d3e7 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:26.476305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 958/1888] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:26 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:32:26.572030 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407577115c3b4744986047c07520f905', 'request_id': 'req-2c80f496-4b41-4019-82c4-ad1602129e8e'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:26.588976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] The token cache is being invalidate because project 407577115c3b4744986047c07520f905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:26.589450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:26.590266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:26.592968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 931/1889] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:26 2022] DELETE /identity/v3/projects/407577115c3b4744986047c07520f905 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:26.599709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776bdde-96ab-4a9c-98ca-c4b72040d3e7 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:26.614200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776bdde-96ab-4a9c-98ca-c4b72040d3e7 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:26.673642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:26.683283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:26.683882 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:26.684179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:26.687203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] PATH_INFO: `/v3/users/118a9620664247cbab3a24dd4cf3bdf1` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:26.687203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] RBAC: Authorizing `identity:delete_user(user_id=118a9620664247cbab3a24dd4cf3bdf1)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:26.687743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:26.753526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:27.014594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118a9620664247cbab3a24dd4cf3bdf1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:27.096344 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118a9620664247cbab3a24dd4cf3bdf1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:27.110744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118a9620664247cbab3a24dd4cf3bdf1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:27.111529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118a9620664247cbab3a24dd4cf3bdf1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:27.120463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-42a70172-6014-4552-8f3f-53d94cc6919a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118a9620664247cbab3a24dd4cf3bdf1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:27.127637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 959/1890] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:26 2022] DELETE /identity/v3/users/118a9620664247cbab3a24dd4cf3bdf1 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:27.916692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:27.918480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:27.935830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80f496-4b41-4019-82c4-ad1602129e8e admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:28.013117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:28.013839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:28.014234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:28.014612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] PATH_INFO: `/v3/projects/9cdd0a766f364a549ec659fab593d5cd` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:28.016867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cdd0a766f364a549ec659fab593d5cd)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:28.018823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:28.231050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cdd0a766f364a549ec659fab593d5cd', 'request_id': 'req-d7de50e9-6df5-479c-a610-cce2d3b462eb'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:28.237278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cdd0a766f364a549ec659fab593d5cd', 'request_id': 'req-d7de50e9-6df5-479c-a610-cce2d3b462eb'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:28.371228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cdd0a766f364a549ec659fab593d5cd', 'request_id': 'req-d7de50e9-6df5-479c-a610-cce2d3b462eb'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:28.391047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] The token cache is being invalidate because project 9cdd0a766f364a549ec659fab593d5cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:28.391369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:28.392098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7de50e9-6df5-479c-a610-cce2d3b462eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:28.394019 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 932/1891] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:27 2022] DELETE /identity/v3/projects/9cdd0a766f364a549ec659fab593d5cd => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:28.399897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c1e2d-591a-4acd-b437-0b7748ab91f0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:28.400372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c1e2d-591a-4acd-b437-0b7748ab91f0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:28.400459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c1e2d-591a-4acd-b437-0b7748ab91f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:28.421787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-958c1e2d-591a-4acd-b437-0b7748ab91f0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:28.427142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-958c1e2d-591a-4acd-b437-0b7748ab91f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:28.493515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 960/1892] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:28.511442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea3a38-5ba7-4965-a310-2a358338539b None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:28.511941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea3a38-5ba7-4965-a310-2a358338539b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:28.512393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea3a38-5ba7-4965-a310-2a358338539b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:28.537254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-aeea3a38-5ba7-4965-a310-2a358338539b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:28.538389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-aeea3a38-5ba7-4965-a310-2a358338539b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:28.547119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 933/1893] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:28.553112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c1e2d-591a-4acd-b437-0b7748ab91f0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:28.570331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c1e2d-591a-4acd-b437-0b7748ab91f0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:28.576402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ffe9-5418-45ba-8e74-fc65a320f0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:28.577057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ffe9-5418-45ba-8e74-fc65a320f0cb admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:28.577340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ffe9-5418-45ba-8e74-fc65a320f0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:28.577612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ffe9-5418-45ba-8e74-fc65a320f0cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:28.578944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ffe9-5418-45ba-8e74-fc65a320f0cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:28.579247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ffe9-5418-45ba-8e74-fc65a320f0cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:28.580928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ffe9-5418-45ba-8e74-fc65a320f0cb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:28.595234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 961/1894] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:32:28.600396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea3a38-5ba7-4965-a310-2a358338539b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:28.609586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea3a38-5ba7-4965-a310-2a358338539b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:28.614465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd4d4c-0712-46f1-ae21-772f9be4c710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:28.615081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd4d4c-0712-46f1-ae21-772f9be4c710 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:28.615361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd4d4c-0712-46f1-ae21-772f9be4c710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:28.615653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd4d4c-0712-46f1-ae21-772f9be4c710 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:28.616261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd4d4c-0712-46f1-ae21-772f9be4c710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:28.618203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd4d4c-0712-46f1-ae21-772f9be4c710 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:28.736528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 934/1895] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:28 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:28.740601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ffe9-5418-45ba-8e74-fc65a320f0cb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:28.768817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ffe9-5418-45ba-8e74-fc65a320f0cb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:28.873729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a4152-e855-4b11-a863-55f82a3adff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:28.874354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a4152-e855-4b11-a863-55f82a3adff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:28.874739 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a4152-e855-4b11-a863-55f82a3adff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:28.875519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a4152-e855-4b11-a863-55f82a3adff0 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:28.876148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a4152-e855-4b11-a863-55f82a3adff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:28.878182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a4152-e855-4b11-a863-55f82a3adff0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:28.947618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:28.994899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 962/1896] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:28 2022] POST /identity/v3/users => generated 291 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:32:29.000059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd4d4c-0712-46f1-ae21-772f9be4c710 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:29.011289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd4d4c-0712-46f1-ae21-772f9be4c710 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:29.017201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:29.019182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378443a2-97d5-4727-96c8-95157e371c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:29.020109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378443a2-97d5-4727-96c8-95157e371c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:29.020480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378443a2-97d5-4727-96c8-95157e371c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:29.020940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378443a2-97d5-4727-96c8-95157e371c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:29.021587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378443a2-97d5-4727-96c8-95157e371c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:29.026916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378443a2-97d5-4727-96c8-95157e371c4c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:29.035119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 935/1897] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:29.039216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a4152-e855-4b11-a863-55f82a3adff0 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:29.051664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a4152-e855-4b11-a863-55f82a3adff0 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:29.056857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65da700-ca68-448e-8eed-5195b1544a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:29.057546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65da700-ca68-448e-8eed-5195b1544a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:29.057888 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65da700-ca68-448e-8eed-5195b1544a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:29.058155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65da700-ca68-448e-8eed-5195b1544a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:29.059034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65da700-ca68-448e-8eed-5195b1544a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:29.060496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65da700-ca68-448e-8eed-5195b1544a70 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:29.073595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 963/1898] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:29.078553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378443a2-97d5-4727-96c8-95157e371c4c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:29.109332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378443a2-97d5-4727-96c8-95157e371c4c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:29.114347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8d7ad-ab16-4cf0-a0f8-f50b0e452b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:29.115188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8d7ad-ab16-4cf0-a0f8-f50b0e452b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:29.115561 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8d7ad-ab16-4cf0-a0f8-f50b0e452b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:29.116067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8d7ad-ab16-4cf0-a0f8-f50b0e452b61 admin admin] PATH_INFO: `/v3/projects/f910065fc9da40798c7f3d4366dab7c6/users/5055ddeab8314c93bb2a664f9d1a7f3a/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:29.134200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8d7ad-ab16-4cf0-a0f8-f50b0e452b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f910065fc9da40798c7f3d4366dab7c6, user_id=5055ddeab8314c93bb2a664f9d1a7f3a, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:29.137006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8d7ad-ab16-4cf0-a0f8-f50b0e452b61 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:29.260754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 936/1899] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:29 2022] PUT /identity/v3/projects/f910065fc9da40798c7f3d4366dab7c6/users/5055ddeab8314c93bb2a664f9d1a7f3a/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:29.267548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbef60-bbf9-4aba-a1d5-59e877f56b35 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:29.268439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbef60-bbf9-4aba-a1d5-59e877f56b35 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:29.269370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbef60-bbf9-4aba-a1d5-59e877f56b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:29.325929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9ebbef60-bbf9-4aba-a1d5-59e877f56b35 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:29.331846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9ebbef60-bbf9-4aba-a1d5-59e877f56b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:29.441893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 964/1900] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:29.457645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8d7ad-ab16-4cf0-a0f8-f50b0e452b61 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:29.459627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-60a8d7ad-ab16-4cf0-a0f8-f50b0e452b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:29.548829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8d7ad-ab16-4cf0-a0f8-f50b0e452b61 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:29.553054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ea676-377a-435b-b225-6d836f63d853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:29.553701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ea676-377a-435b-b225-6d836f63d853 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:29.554625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ea676-377a-435b-b225-6d836f63d853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:29.554901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ea676-377a-435b-b225-6d836f63d853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:29.566065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ea676-377a-435b-b225-6d836f63d853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:29.567532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ea676-377a-435b-b225-6d836f63d853 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:29.577328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 937/1901] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:32:31.397932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbef60-bbf9-4aba-a1d5-59e877f56b35 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:31.399545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9ebbef60-bbf9-4aba-a1d5-59e877f56b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:31.412380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1289ea-6957-4780-82c8-fe7b9b3b7a0f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:31.412964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1289ea-6957-4780-82c8-fe7b9b3b7a0f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:31.413184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1289ea-6957-4780-82c8-fe7b9b3b7a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:31.539877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-af1289ea-6957-4780-82c8-fe7b9b3b7a0f None None] MFA Rules not processed for user `5055ddeab8314c93bb2a664f9d1a7f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:31.541154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-af1289ea-6957-4780-82c8-fe7b9b3b7a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:31.541465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbef60-bbf9-4aba-a1d5-59e877f56b35 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:31.545439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e1866-87eb-42a7-ba52-1877eb97e20a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:31.546542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e1866-87eb-42a7-ba52-1877eb97e20a service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:31.547248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e1866-87eb-42a7-ba52-1877eb97e20a service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:31.547876 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e1866-87eb-42a7-ba52-1877eb97e20a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:31.550155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6d3e1866-87eb-42a7-ba52-1877eb97e20a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:31.644894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e1866-87eb-42a7-ba52-1877eb97e20a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:31.646439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e1866-87eb-42a7-ba52-1877eb97e20a service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:31.647072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 938/1902] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:31.666213 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 965/1903] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:31.670673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1289ea-6957-4780-82c8-fe7b9b3b7a0f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:31.674055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-af1289ea-6957-4780-82c8-fe7b9b3b7a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:31.690727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1289ea-6957-4780-82c8-fe7b9b3b7a0f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:31.692825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:31.696889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e12f8-9e09-4657-8687-da2987d4e128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:31.697913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e12f8-9e09-4657-8687-da2987d4e128 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:31.698433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e12f8-9e09-4657-8687-da2987d4e128 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:31.698954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e12f8-9e09-4657-8687-da2987d4e128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:31.713108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e12f8-9e09-4657-8687-da2987d4e128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:31.715152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e12f8-9e09-4657-8687-da2987d4e128 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:31.727666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 939/1904] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:31.742677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:33.776309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:33.810324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:34.621607 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:34.677558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:35.158931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:35.409105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:36.113223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:36.300335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:36.717684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:36.747250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:37.159124 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:37.466364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:37.855855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:38.136703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:38.159091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:38.822190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:39.479720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:39.583127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:39.915620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433396ff-b651-4e6f-be7e-6d84164e8dee None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:39.916101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433396ff-b651-4e6f-be7e-6d84164e8dee None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:39.917129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433396ff-b651-4e6f-be7e-6d84164e8dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:39.964654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-433396ff-b651-4e6f-be7e-6d84164e8dee None None] MFA Rules not processed for user `5055ddeab8314c93bb2a664f9d1a7f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:39.967372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-433396ff-b651-4e6f-be7e-6d84164e8dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:39.987519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 966/1905] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:40.702202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:41.136840 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:43.347481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:43.628864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:44.222947 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e12f8-9e09-4657-8687-da2987d4e128 service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:44.225558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0d5e12f8-9e09-4657-8687-da2987d4e128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:44.261015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e12f8-9e09-4657-8687-da2987d4e128 service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:44.269714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:44.270500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:44.270886 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:44.271410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] PATH_INFO: `/v3/users/b87a5fc6155a41fa8d7cd61762d24278` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:44.275525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87a5fc6155a41fa8d7cd61762d24278)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:44.278177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:44.621729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87a5fc6155a41fa8d7cd61762d24278'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:44.623281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87a5fc6155a41fa8d7cd61762d24278'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:44.624703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87a5fc6155a41fa8d7cd61762d24278'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:44.655367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87a5fc6155a41fa8d7cd61762d24278'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:44.687933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87a5fc6155a41fa8d7cd61762d24278'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:44.768131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 940/1906] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:44 2022] DELETE /identity/v3/users/b87a5fc6155a41fa8d7cd61762d24278 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:44.773879 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b01343-8de2-4360-b24b-79653f7d6283 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:44.774639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b01343-8de2-4360-b24b-79653f7d6283 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:44.774639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b01343-8de2-4360-b24b-79653f7d6283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:44.821397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-58b01343-8de2-4360-b24b-79653f7d6283 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:44.823025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-58b01343-8de2-4360-b24b-79653f7d6283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:44.917710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 967/1907] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:44.929908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:44.931474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:45.015593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a42899-bcc8-43f8-bf39-284134f3c9ce admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:45.040686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b918a9e-cdf4-44c4-aaa6-931a1e996cbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:45.043374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b918a9e-cdf4-44c4-aaa6-931a1e996cbf service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:45.048874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b918a9e-cdf4-44c4-aaa6-931a1e996cbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:45.049714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b918a9e-cdf4-44c4-aaa6-931a1e996cbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:45.077588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:45.087665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b918a9e-cdf4-44c4-aaa6-931a1e996cbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:45.090361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b918a9e-cdf4-44c4-aaa6-931a1e996cbf service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:45.115023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 941/1908] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:46.158507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b01343-8de2-4360-b24b-79653f7d6283 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:46.160896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-58b01343-8de2-4360-b24b-79653f7d6283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:46.186129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b01343-8de2-4360-b24b-79653f7d6283 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:46.197605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:46.198263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:46.198561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:46.198927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] PATH_INFO: `/v3/projects/34e391f0088d4ff7b85efa0419e3057e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:46.200156 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e391f0088d4ff7b85efa0419e3057e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:46.202724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:46.210828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b918a9e-cdf4-44c4-aaa6-931a1e996cbf service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:46.213068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9b918a9e-cdf4-44c4-aaa6-931a1e996cbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:46.231289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b918a9e-cdf4-44c4-aaa6-931a1e996cbf service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:46.238140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:46.238781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:46.254492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:46.254492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] PATH_INFO: `/v3/users/079ca11e098645538fbc708c979c4dca` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:46.254492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] RBAC: Authorizing `identity:delete_user(user_id=079ca11e098645538fbc708c979c4dca)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:46.254492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:46.350228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:46.368878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e391f0088d4ff7b85efa0419e3057e', 'request_id': 'req-e0c73177-2719-4611-9f26-6e8d2e5cb492'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.377924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e391f0088d4ff7b85efa0419e3057e', 'request_id': 'req-e0c73177-2719-4611-9f26-6e8d2e5cb492'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.456161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e391f0088d4ff7b85efa0419e3057e', 'request_id': 'req-e0c73177-2719-4611-9f26-6e8d2e5cb492'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.475621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] The token cache is being invalidate because project 34e391f0088d4ff7b85efa0419e3057e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:46.476087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.477454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.477664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '079ca11e098645538fbc708c979c4dca'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.478531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '079ca11e098645538fbc708c979c4dca'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.479318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '079ca11e098645538fbc708c979c4dca'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.486824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 968/1909] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:46 2022] DELETE /identity/v3/projects/34e391f0088d4ff7b85efa0419e3057e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:46.488562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:46.492456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:46.495645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '079ca11e098645538fbc708c979c4dca'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.517911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1e94a07c-e21a-4485-acf4-76a6391d265f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '079ca11e098645538fbc708c979c4dca'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:46.576479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 942/1910] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:46 2022] DELETE /identity/v3/users/079ca11e098645538fbc708c979c4dca => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:46.582801 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37a3a0-420b-4cdc-b8da-323ead286046 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:46.583101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37a3a0-420b-4cdc-b8da-323ead286046 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:46.583341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37a3a0-420b-4cdc-b8da-323ead286046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:46.598488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c73177-2719-4611-9f26-6e8d2e5cb492 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:46.623460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cc37a3a0-420b-4cdc-b8da-323ead286046 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:46.625452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cc37a3a0-420b-4cdc-b8da-323ead286046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:46.722596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:46.723257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:46.723534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:46.723809 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] PATH_INFO: `/v3/users/cd13d8419d3d44c1ad371979d722406b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:46.724980 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd13d8419d3d44c1ad371979d722406b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:46.727217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:46.727997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 943/1911] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:46.743574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37a3a0-420b-4cdc-b8da-323ead286046 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:46.745352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cc37a3a0-420b-4cdc-b8da-323ead286046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:46.829625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37a3a0-420b-4cdc-b8da-323ead286046 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:46.833619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0144580-418a-44c8-95bd-601ecbe4133e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:46.834240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0144580-418a-44c8-95bd-601ecbe4133e service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:46.834537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0144580-418a-44c8-95bd-601ecbe4133e service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:46.834877 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0144580-418a-44c8-95bd-601ecbe4133e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:46.849692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0144580-418a-44c8-95bd-601ecbe4133e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:46.851458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0144580-418a-44c8-95bd-601ecbe4133e service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:46.866586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 944/1912] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:46.909671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd13d8419d3d44c1ad371979d722406b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:47.045112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd13d8419d3d44c1ad371979d722406b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:47.058338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:47.075441 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd13d8419d3d44c1ad371979d722406b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:47.076656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd13d8419d3d44c1ad371979d722406b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:47.088613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd13d8419d3d44c1ad371979d722406b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:47.097071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 969/1913] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:46 2022] DELETE /identity/v3/users/cd13d8419d3d44c1ad371979d722406b => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:47.570905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0144580-418a-44c8-95bd-601ecbe4133e service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:47.572410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b0144580-418a-44c8-95bd-601ecbe4133e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:47.586517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0144580-418a-44c8-95bd-601ecbe4133e service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:47.602900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:47.707607 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:47.708454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:47.709026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:47.709463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] PATH_INFO: `/v3/projects/7c0f847c4bb74cc69b012a945ab806e1` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:47.710916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c0f847c4bb74cc69b012a945ab806e1)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:47.712860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:47.744764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:47.758669 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:47.784560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6b171-d9b1-49a1-ba64-488284b40fcb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:47.790501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:47.791328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:47.791781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:47.792224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] PATH_INFO: `/v3/projects/e929fdf7aedf429fa61ed39cdb5f1991` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:47.794144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e929fdf7aedf429fa61ed39cdb5f1991)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:47.796638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:47.870852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c0f847c4bb74cc69b012a945ab806e1', 'request_id': 'req-e9309c19-dc36-4305-a679-6f3cf7f7ff13'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:47.878038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c0f847c4bb74cc69b012a945ab806e1', 'request_id': 'req-e9309c19-dc36-4305-a679-6f3cf7f7ff13'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.005390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e929fdf7aedf429fa61ed39cdb5f1991', 'request_id': 'req-30516337-71de-413a-a426-d27ff9c9a472'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.008516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c0f847c4bb74cc69b012a945ab806e1', 'request_id': 'req-e9309c19-dc36-4305-a679-6f3cf7f7ff13'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.019354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e929fdf7aedf429fa61ed39cdb5f1991', 'request_id': 'req-30516337-71de-413a-a426-d27ff9c9a472'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.022333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] The token cache is being invalidate because project 7c0f847c4bb74cc69b012a945ab806e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:48.022679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.024093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.026697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 945/1914] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:47 2022] DELETE /identity/v3/projects/7c0f847c4bb74cc69b012a945ab806e1 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:48.033987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:48.035739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:48.112485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e929fdf7aedf429fa61ed39cdb5f1991', 'request_id': 'req-30516337-71de-413a-a426-d27ff9c9a472'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.125513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] The token cache is being invalidate because project e929fdf7aedf429fa61ed39cdb5f1991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:48.125806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.126470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-30516337-71de-413a-a426-d27ff9c9a472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.128398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 970/1915] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:47 2022] DELETE /identity/v3/projects/e929fdf7aedf429fa61ed39cdb5f1991 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:48.136147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a98233-6cc6-4d3f-be8c-75ecf9ffdb80 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:48.137226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a98233-6cc6-4d3f-be8c-75ecf9ffdb80 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:48.137761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a98233-6cc6-4d3f-be8c-75ecf9ffdb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:48.139430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9309c19-dc36-4305-a679-6f3cf7f7ff13 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:48.162487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-61a98233-6cc6-4d3f-be8c-75ecf9ffdb80 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:48.163735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-61a98233-6cc6-4d3f-be8c-75ecf9ffdb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:48.203935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:48.227621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:48.228518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:48.229206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:48.229588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] PATH_INFO: `/v3/users/0107f87945fd43d18862b8136f90279f` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:48.231224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0107f87945fd43d18862b8136f90279f)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:48.234323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:48.248530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 971/1916] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:48.264130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5a03b-f449-4d0b-922f-6526e83a375c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:48.265262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5a03b-f449-4d0b-922f-6526e83a375c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:48.265709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5a03b-f449-4d0b-922f-6526e83a375c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:48.293585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a8c5a03b-f449-4d0b-922f-6526e83a375c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:48.294834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a8c5a03b-f449-4d0b-922f-6526e83a375c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:48.304413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 972/1917] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:48.311413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5a03b-f449-4d0b-922f-6526e83a375c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:48.322666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5a03b-f449-4d0b-922f-6526e83a375c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:48.328399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccc6ea-4689-46be-b475-0b10a359f0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:48.329071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccc6ea-4689-46be-b475-0b10a359f0ca admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:48.329346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccc6ea-4689-46be-b475-0b10a359f0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:48.329614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccc6ea-4689-46be-b475-0b10a359f0ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:48.330266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccc6ea-4689-46be-b475-0b10a359f0ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:48.330590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccc6ea-4689-46be-b475-0b10a359f0ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:48.332204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccc6ea-4689-46be-b475-0b10a359f0ca admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:48.348269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 973/1918] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:32:48.352361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccc6ea-4689-46be-b475-0b10a359f0ca admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:48.364443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccc6ea-4689-46be-b475-0b10a359f0ca admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:48.369118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e14d21-64d9-446f-9fe4-a6779c4826e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:48.370124 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e14d21-64d9-446f-9fe4-a6779c4826e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:48.370407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e14d21-64d9-446f-9fe4-a6779c4826e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:48.370679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e14d21-64d9-446f-9fe4-a6779c4826e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:48.371294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e14d21-64d9-446f-9fe4-a6779c4826e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:48.373224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e14d21-64d9-446f-9fe4-a6779c4826e6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:48.461153 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0107f87945fd43d18862b8136f90279f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.462089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0107f87945fd43d18862b8136f90279f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.462980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0107f87945fd43d18862b8136f90279f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.472284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0107f87945fd43d18862b8136f90279f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.487920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0107f87945fd43d18862b8136f90279f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:48.514284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 974/1919] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:48 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:48.519853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e14d21-64d9-446f-9fe4-a6779c4826e6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:48.521806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-22e14d21-64d9-446f-9fe4-a6779c4826e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:48.560209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:48.581767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 946/1920] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:48 2022] DELETE /identity/v3/users/0107f87945fd43d18862b8136f90279f => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:48.620613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e14d21-64d9-446f-9fe4-a6779c4826e6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:48.716193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90293de0-2809-4a60-8f82-fcbf4533534a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:48.716193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90293de0-2809-4a60-8f82-fcbf4533534a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:48.716193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90293de0-2809-4a60-8f82-fcbf4533534a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:48.716193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90293de0-2809-4a60-8f82-fcbf4533534a admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:48.718145 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90293de0-2809-4a60-8f82-fcbf4533534a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:48.720490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90293de0-2809-4a60-8f82-fcbf4533534a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:48.823681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 975/1921] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:48 2022] POST /identity/v3/users => generated 286 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:32:48.830948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:48.849964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0efa0d-6321-4db6-81a1-cbfe582f9b89 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:48.859450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb919333-2d4b-4091-b73b-9a1d60ca18b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:48.861847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb919333-2d4b-4091-b73b-9a1d60ca18b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:48.876435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb919333-2d4b-4091-b73b-9a1d60ca18b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:48.877143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb919333-2d4b-4091-b73b-9a1d60ca18b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:48.878486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb919333-2d4b-4091-b73b-9a1d60ca18b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:48.887513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb919333-2d4b-4091-b73b-9a1d60ca18b3 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:48.915725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 947/1922] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:48.920275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90293de0-2809-4a60-8f82-fcbf4533534a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:48.931503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90293de0-2809-4a60-8f82-fcbf4533534a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:48.937533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dd3fc-f15f-47f0-b4cd-e5205e3dc9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:48.938380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dd3fc-f15f-47f0-b4cd-e5205e3dc9cc admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:48.938756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dd3fc-f15f-47f0-b4cd-e5205e3dc9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:48.939282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dd3fc-f15f-47f0-b4cd-e5205e3dc9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:48.940170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dd3fc-f15f-47f0-b4cd-e5205e3dc9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:48.942192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dd3fc-f15f-47f0-b4cd-e5205e3dc9cc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:48.959541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 976/1923] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:48.960910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb919333-2d4b-4091-b73b-9a1d60ca18b3 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:48.972011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb919333-2d4b-4091-b73b-9a1d60ca18b3 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:48.976438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68727e2-ff9e-4831-8110-d0b6d2ca35d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:48.977123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68727e2-ff9e-4831-8110-d0b6d2ca35d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:48.977424 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68727e2-ff9e-4831-8110-d0b6d2ca35d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:48.977716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68727e2-ff9e-4831-8110-d0b6d2ca35d0 admin admin] PATH_INFO: `/v3/projects/a66dd577cfe8414bb4bc31c42db72e90/users/a48caf30ea4940998a692c71230378ed/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:49.000727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68727e2-ff9e-4831-8110-d0b6d2ca35d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66dd577cfe8414bb4bc31c42db72e90, user_id=a48caf30ea4940998a692c71230378ed, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:49.007415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68727e2-ff9e-4831-8110-d0b6d2ca35d0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:49.152088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 948/1924] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:48 2022] PUT /identity/v3/projects/a66dd577cfe8414bb4bc31c42db72e90/users/a48caf30ea4940998a692c71230378ed/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:49.158072 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd495654-4771-4608-8ce9-5d115c8318cd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:49.158390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd495654-4771-4608-8ce9-5d115c8318cd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:49.158668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd495654-4771-4608-8ce9-5d115c8318cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:49.181398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-dd495654-4771-4608-8ce9-5d115c8318cd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:49.183279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dd495654-4771-4608-8ce9-5d115c8318cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:49.202680 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68727e2-ff9e-4831-8110-d0b6d2ca35d0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:49.206192 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e68727e2-ff9e-4831-8110-d0b6d2ca35d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:49.340535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 977/1925] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:49.340858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68727e2-ff9e-4831-8110-d0b6d2ca35d0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:49.348895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd495654-4771-4608-8ce9-5d115c8318cd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:49.350526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-dd495654-4771-4608-8ce9-5d115c8318cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:49.360228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:49.361123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:49.361992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:49.362420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] PATH_INFO: `/v3/projects/49f27d7caa1f4c01bbf20020baac5ba3` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:49.364095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f27d7caa1f4c01bbf20020baac5ba3)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:49.368825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:49.453418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd495654-4771-4608-8ce9-5d115c8318cd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:49.457939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29edd928-5cb1-4775-900f-9d1c4b80c3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:49.458674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edd928-5cb1-4775-900f-9d1c4b80c3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:49.459053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edd928-5cb1-4775-900f-9d1c4b80c3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:49.459459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29edd928-5cb1-4775-900f-9d1c4b80c3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:49.466748 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:49.472448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edd928-5cb1-4775-900f-9d1c4b80c3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:49.475212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29edd928-5cb1-4775-900f-9d1c4b80c3b9 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:49.493997 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 978/1926] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:32:49.513508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:49.535471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f27d7caa1f4c01bbf20020baac5ba3', 'request_id': 'req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:49.535471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f27d7caa1f4c01bbf20020baac5ba3', 'request_id': 'req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:49.611673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f27d7caa1f4c01bbf20020baac5ba3', 'request_id': 'req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:49.636525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] The token cache is being invalidate because project 49f27d7caa1f4c01bbf20020baac5ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:49.637299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:49.638648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4ebfa5c2-2e6c-4f8a-9dc7-aeae9be35bd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:49.642143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 949/1927] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:49 2022] DELETE /identity/v3/projects/49f27d7caa1f4c01bbf20020baac5ba3 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:49.659627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26fbea-653d-4b0b-ad5b-b2eae1cf1249 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:49.662475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26fbea-653d-4b0b-ad5b-b2eae1cf1249 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:49.662475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26fbea-653d-4b0b-ad5b-b2eae1cf1249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:49.692165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5f26fbea-653d-4b0b-ad5b-b2eae1cf1249 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:49.695310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5f26fbea-653d-4b0b-ad5b-b2eae1cf1249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:49.833740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 979/1928] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:49.853865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f3abb-46be-426b-bd26-8f977b4429a8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:49.854172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f3abb-46be-426b-bd26-8f977b4429a8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:49.854413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f3abb-46be-426b-bd26-8f977b4429a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:49.881130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-049f3abb-46be-426b-bd26-8f977b4429a8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:49.884390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-049f3abb-46be-426b-bd26-8f977b4429a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:49.897274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 950/1929] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:49.903589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26fbea-653d-4b0b-ad5b-b2eae1cf1249 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:49.916130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26fbea-653d-4b0b-ad5b-b2eae1cf1249 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:49.922640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229f95c-8a6a-4a6e-9b56-29a4baebc514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:49.923574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229f95c-8a6a-4a6e-9b56-29a4baebc514 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:49.923863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229f95c-8a6a-4a6e-9b56-29a4baebc514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:49.924128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229f95c-8a6a-4a6e-9b56-29a4baebc514 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:49.924844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229f95c-8a6a-4a6e-9b56-29a4baebc514 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:49.925407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229f95c-8a6a-4a6e-9b56-29a4baebc514 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:49.927262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229f95c-8a6a-4a6e-9b56-29a4baebc514 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:49.960838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 980/1930] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:32:49.967843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f3abb-46be-426b-bd26-8f977b4429a8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:49.976479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f3abb-46be-426b-bd26-8f977b4429a8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:49.982405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c95a8a-2d57-4920-a003-da0045f9a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:49.983031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c95a8a-2d57-4920-a003-da0045f9a3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:49.983342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c95a8a-2d57-4920-a003-da0045f9a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:49.984320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c95a8a-2d57-4920-a003-da0045f9a3c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:49.985007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c95a8a-2d57-4920-a003-da0045f9a3c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:49.988437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c95a8a-2d57-4920-a003-da0045f9a3c1 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:50.137231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 951/1931] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:49 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:50.144230 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229f95c-8a6a-4a6e-9b56-29a4baebc514 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:50.155986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229f95c-8a6a-4a6e-9b56-29a4baebc514 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:50.235154 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f48a2-1d86-4ec7-b9b3-e40dd99e1906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:50.236097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f48a2-1d86-4ec7-b9b3-e40dd99e1906 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:50.236496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f48a2-1d86-4ec7-b9b3-e40dd99e1906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:50.236917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f48a2-1d86-4ec7-b9b3-e40dd99e1906 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:50.237873 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f48a2-1d86-4ec7-b9b3-e40dd99e1906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:50.240883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f48a2-1d86-4ec7-b9b3-e40dd99e1906 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:50.466480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 981/1932] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:50 2022] POST /identity/v3/users => generated 293 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:50.472386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c95a8a-2d57-4920-a003-da0045f9a3c1 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:50.493615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c95a8a-2d57-4920-a003-da0045f9a3c1 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:50.503318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5952e1-0b33-4a13-b185-9593b33612f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:50.504897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5952e1-0b33-4a13-b185-9593b33612f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:50.505500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5952e1-0b33-4a13-b185-9593b33612f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:50.505940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5952e1-0b33-4a13-b185-9593b33612f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:50.506982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5952e1-0b33-4a13-b185-9593b33612f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:50.509844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5952e1-0b33-4a13-b185-9593b33612f2 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:50.531907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 952/1933] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:50.538079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f48a2-1d86-4ec7-b9b3-e40dd99e1906 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:50.574974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f48a2-1d86-4ec7-b9b3-e40dd99e1906 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:50.588641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fd03c-6510-407a-b93c-2fdec1e37b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:50.589654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fd03c-6510-407a-b93c-2fdec1e37b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:50.590138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fd03c-6510-407a-b93c-2fdec1e37b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:50.590869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fd03c-6510-407a-b93c-2fdec1e37b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:50.591743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fd03c-6510-407a-b93c-2fdec1e37b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:50.596490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fd03c-6510-407a-b93c-2fdec1e37b24 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:50.612764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 982/1934] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:32:50.619902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5952e1-0b33-4a13-b185-9593b33612f2 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:50.634459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5952e1-0b33-4a13-b185-9593b33612f2 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:50.641214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66d2e9-67a5-4814-bd3d-3e4b322b9a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:50.642179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66d2e9-67a5-4814-bd3d-3e4b322b9a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:50.642611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66d2e9-67a5-4814-bd3d-3e4b322b9a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:50.643115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66d2e9-67a5-4814-bd3d-3e4b322b9a8d admin admin] PATH_INFO: `/v3/projects/00144386aca044b9b365ceac5ea10792/users/2692f6209b684a3daa7d442bd957106c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:50.669722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66d2e9-67a5-4814-bd3d-3e4b322b9a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=00144386aca044b9b365ceac5ea10792, user_id=2692f6209b684a3daa7d442bd957106c, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:50.675688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66d2e9-67a5-4814-bd3d-3e4b322b9a8d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:50.789331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 953/1935] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:50 2022] PUT /identity/v3/projects/00144386aca044b9b365ceac5ea10792/users/2692f6209b684a3daa7d442bd957106c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:32:50.806440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74b738-0833-4570-9427-c2f55552859b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:50.806990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74b738-0833-4570-9427-c2f55552859b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:50.807520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74b738-0833-4570-9427-c2f55552859b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:50.890444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6a74b738-0833-4570-9427-c2f55552859b None None] MFA Rules not processed for user `2692f6209b684a3daa7d442bd957106c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:50.901047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6a74b738-0833-4570-9427-c2f55552859b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:50.999471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:51.069163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 983/1936] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:51.085874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66d2e9-67a5-4814-bd3d-3e4b322b9a8d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:51.088725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6b66d2e9-67a5-4814-bd3d-3e4b322b9a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:51.218279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66d2e9-67a5-4814-bd3d-3e4b322b9a8d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:51.230472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3835155-936e-475a-acef-235bbe2e26ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:51.236900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3835155-936e-475a-acef-235bbe2e26ef service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:51.236900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3835155-936e-475a-acef-235bbe2e26ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:51.236900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3835155-936e-475a-acef-235bbe2e26ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:51.250432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3835155-936e-475a-acef-235bbe2e26ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:51.253366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3835155-936e-475a-acef-235bbe2e26ef service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:51.282319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 954/1937] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:51.305336 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74b738-0833-4570-9427-c2f55552859b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:51.406772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74b738-0833-4570-9427-c2f55552859b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:51.411643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:51.412288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:51.412566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:51.412890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] PATH_INFO: `/v3/users/2692f6209b684a3daa7d442bd957106c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:51.414125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2692f6209b684a3daa7d442bd957106c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:51.416083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:51.625295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab59268-d671-4cdc-831d-426bc2ec728d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:51.625864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab59268-d671-4cdc-831d-426bc2ec728d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:51.626210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab59268-d671-4cdc-831d-426bc2ec728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:51.685290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2692f6209b684a3daa7d442bd957106c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:51.760707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cab59268-d671-4cdc-831d-426bc2ec728d None None] MFA Rules not processed for user `a48caf30ea4940998a692c71230378ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:51.763521 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cab59268-d671-4cdc-831d-426bc2ec728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:51.778750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2692f6209b684a3daa7d442bd957106c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:51.821068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2692f6209b684a3daa7d442bd957106c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:51.825275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2692f6209b684a3daa7d442bd957106c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:51.859195 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-27412d35-7f30-404b-a57f-5775043c97e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2692f6209b684a3daa7d442bd957106c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:51.884491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 984/1938] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:32:51 2022] DELETE /identity/v3/users/2692f6209b684a3daa7d442bd957106c => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:51.900626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead1985-3a17-4f7e-8b89-88d764fd2315 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:51.901913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead1985-3a17-4f7e-8b89-88d764fd2315 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:51.901913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead1985-3a17-4f7e-8b89-88d764fd2315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:51.969992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-bead1985-3a17-4f7e-8b89-88d764fd2315 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:51.972087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bead1985-3a17-4f7e-8b89-88d764fd2315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:51.996216 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:52.045438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 955/1939] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:52.064478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab59268-d671-4cdc-831d-426bc2ec728d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:52.065598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cab59268-d671-4cdc-831d-426bc2ec728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:52.114883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab59268-d671-4cdc-831d-426bc2ec728d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:52.126960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d74f7-31bb-4116-8f24-cd7a376baac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:52.127982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d74f7-31bb-4116-8f24-cd7a376baac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:52.128454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d74f7-31bb-4116-8f24-cd7a376baac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:52.128977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d74f7-31bb-4116-8f24-cd7a376baac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:52.130087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d74f7-31bb-4116-8f24-cd7a376baac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:52.130567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 985/1940] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:52.134067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d74f7-31bb-4116-8f24-cd7a376baac2 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:52.153634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead1985-3a17-4f7e-8b89-88d764fd2315 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:52.153634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bead1985-3a17-4f7e-8b89-88d764fd2315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:52.171939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:52.243592 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 956/1941] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:52 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:52.256292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d74f7-31bb-4116-8f24-cd7a376baac2 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:52.303152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead1985-3a17-4f7e-8b89-88d764fd2315 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:52.395049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d74f7-31bb-4116-8f24-cd7a376baac2 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:52.401665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86b9b7-ccec-44d1-8fd9-312812fd1d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:52.402285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86b9b7-ccec-44d1-8fd9-312812fd1d4c admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:52.402742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86b9b7-ccec-44d1-8fd9-312812fd1d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:52.403122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86b9b7-ccec-44d1-8fd9-312812fd1d4c admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:52.404344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86b9b7-ccec-44d1-8fd9-312812fd1d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:52.406726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86b9b7-ccec-44d1-8fd9-312812fd1d4c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:52.414729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7953e1e-eb34-4239-a15a-60eb812d4296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:52.417032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953e1e-eb34-4239-a15a-60eb812d4296 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:52.417930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953e1e-eb34-4239-a15a-60eb812d4296 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:52.418675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7953e1e-eb34-4239-a15a-60eb812d4296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:52.433136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7953e1e-eb34-4239-a15a-60eb812d4296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:52.435931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7953e1e-eb34-4239-a15a-60eb812d4296 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:52.446833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 986/1942] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:52.522410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 957/1943] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:52 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:52.529224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7953e1e-eb34-4239-a15a-60eb812d4296 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:52.563686 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7953e1e-eb34-4239-a15a-60eb812d4296 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:52.571797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a532620-2da4-4789-b56e-a5b2282e3aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:52.572772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a532620-2da4-4789-b56e-a5b2282e3aeb admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:52.573462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a532620-2da4-4789-b56e-a5b2282e3aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:52.574219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a532620-2da4-4789-b56e-a5b2282e3aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:52.575448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a532620-2da4-4789-b56e-a5b2282e3aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:52.579885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a532620-2da4-4789-b56e-a5b2282e3aeb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:52.598497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 987/1944] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:52.602893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86b9b7-ccec-44d1-8fd9-312812fd1d4c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:52.620893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86b9b7-ccec-44d1-8fd9-312812fd1d4c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:52.629983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb201f64-b652-410e-a76c-787a71d2cf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:52.631229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb201f64-b652-410e-a76c-787a71d2cf9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:52.631716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb201f64-b652-410e-a76c-787a71d2cf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:52.632189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb201f64-b652-410e-a76c-787a71d2cf9b admin admin] PATH_INFO: `/v3/projects/88cbb5cd9b664838a3c70e4017c54ba9/users/0cf6efaa09a440d0a17564c9d48196f3/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:52.662470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb201f64-b652-410e-a76c-787a71d2cf9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=88cbb5cd9b664838a3c70e4017c54ba9, user_id=0cf6efaa09a440d0a17564c9d48196f3, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:52.667186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb201f64-b652-410e-a76c-787a71d2cf9b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:52.796947 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 958/1945] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:52 2022] PUT /identity/v3/projects/88cbb5cd9b664838a3c70e4017c54ba9/users/0cf6efaa09a440d0a17564c9d48196f3/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:32:52.799662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a532620-2da4-4789-b56e-a5b2282e3aeb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:52.814447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a532620-2da4-4789-b56e-a5b2282e3aeb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:52.929089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be7b5f-8874-4d2d-a8cf-d3d56b1135b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:52.930253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be7b5f-8874-4d2d-a8cf-d3d56b1135b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:52.930732 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be7b5f-8874-4d2d-a8cf-d3d56b1135b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:52.931392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be7b5f-8874-4d2d-a8cf-d3d56b1135b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:52.932658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be7b5f-8874-4d2d-a8cf-d3d56b1135b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:52.935779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be7b5f-8874-4d2d-a8cf-d3d56b1135b8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:52.956024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 988/1946] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:32:52.962516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb201f64-b652-410e-a76c-787a71d2cf9b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:52.982581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb201f64-b652-410e-a76c-787a71d2cf9b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:52.990417 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b82ea8-d738-4b8b-a873-328922a5420d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:52.991522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b82ea8-d738-4b8b-a873-328922a5420d admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:52.991996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b82ea8-d738-4b8b-a873-328922a5420d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:52.992461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b82ea8-d738-4b8b-a873-328922a5420d admin admin] PATH_INFO: `/v3/projects/88cbb5cd9b664838a3c70e4017c54ba9/users/0cf6efaa09a440d0a17564c9d48196f3/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:52.996154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b82ea8-d738-4b8b-a873-328922a5420d admin admin] RBAC: Authorizing `identity:create_grant(project_id=88cbb5cd9b664838a3c70e4017c54ba9, user_id=0cf6efaa09a440d0a17564c9d48196f3, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:53.001949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b82ea8-d738-4b8b-a873-328922a5420d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:53.021134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 959/1947] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:52 2022] PUT /identity/v3/projects/88cbb5cd9b664838a3c70e4017c54ba9/users/0cf6efaa09a440d0a17564c9d48196f3/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:53.387327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be7b5f-8874-4d2d-a8cf-d3d56b1135b8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:53.391242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-48be7b5f-8874-4d2d-a8cf-d3d56b1135b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:53.472285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be7b5f-8874-4d2d-a8cf-d3d56b1135b8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:53.558432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:53.574787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:53.575576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:53.575934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:53.576262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] PATH_INFO: `/v3/projects/00144386aca044b9b365ceac5ea10792` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:53.579902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] RBAC: Authorizing `identity:delete_project(project_id=00144386aca044b9b365ceac5ea10792)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:53.583320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:53.788597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00144386aca044b9b365ceac5ea10792', 'request_id': 'req-cf205307-6309-49f1-85c5-cda1a75ff09a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:53.799457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00144386aca044b9b365ceac5ea10792', 'request_id': 'req-cf205307-6309-49f1-85c5-cda1a75ff09a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:53.902903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00144386aca044b9b365ceac5ea10792', 'request_id': 'req-cf205307-6309-49f1-85c5-cda1a75ff09a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:53.923804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] The token cache is being invalidate because project 00144386aca044b9b365ceac5ea10792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:32:53.924444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:53.925508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf205307-6309-49f1-85c5-cda1a75ff09a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:32:53.928546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 989/1948] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:32:53 2022] DELETE /identity/v3/projects/00144386aca044b9b365ceac5ea10792 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:53.936965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fb3db-8cb6-4abb-ad90-40c87c6ad7ad None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:53.937486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fb3db-8cb6-4abb-ad90-40c87c6ad7ad None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:53.939828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fb3db-8cb6-4abb-ad90-40c87c6ad7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:53.964069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6b4fb3db-8cb6-4abb-ad90-40c87c6ad7ad None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:53.965696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6b4fb3db-8cb6-4abb-ad90-40c87c6ad7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:54.111744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 960/1949] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:54.125915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9c952-d03b-474e-953b-09e5e286f90b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:54.126635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9c952-d03b-474e-953b-09e5e286f90b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:54.127138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9c952-d03b-474e-953b-09e5e286f90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:54.161787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c1d9c952-d03b-474e-953b-09e5e286f90b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:54.163823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c1d9c952-d03b-474e-953b-09e5e286f90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:54.188329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 990/1950] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:54.195395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fb3db-8cb6-4abb-ad90-40c87c6ad7ad None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:54.219992 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fb3db-8cb6-4abb-ad90-40c87c6ad7ad None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:54.229768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af23d95-3fcd-40aa-b165-7cc8affa5278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:54.235795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af23d95-3fcd-40aa-b165-7cc8affa5278 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:54.235795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af23d95-3fcd-40aa-b165-7cc8affa5278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:54.235795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af23d95-3fcd-40aa-b165-7cc8affa5278 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:54.235795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af23d95-3fcd-40aa-b165-7cc8affa5278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:32:54.236357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af23d95-3fcd-40aa-b165-7cc8affa5278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:54.240132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af23d95-3fcd-40aa-b165-7cc8affa5278 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:54.281816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 961/1951] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:32:54.286812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9c952-d03b-474e-953b-09e5e286f90b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:54.310119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9c952-d03b-474e-953b-09e5e286f90b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:54.318887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80764b1d-a2cb-4afa-acc6-e17fc7baa075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:54.319908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80764b1d-a2cb-4afa-acc6-e17fc7baa075 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:54.320372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80764b1d-a2cb-4afa-acc6-e17fc7baa075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:54.320879 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80764b1d-a2cb-4afa-acc6-e17fc7baa075 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:54.322030 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80764b1d-a2cb-4afa-acc6-e17fc7baa075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:54.325731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80764b1d-a2cb-4afa-acc6-e17fc7baa075 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:54.509362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 991/1952] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:32:54 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:54.514748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af23d95-3fcd-40aa-b165-7cc8affa5278 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:54.525495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af23d95-3fcd-40aa-b165-7cc8affa5278 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:54.539829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:54.663447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d39a59-b78a-4cf6-91a5-22392bc6773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:54.664919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d39a59-b78a-4cf6-91a5-22392bc6773d admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:54.665749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d39a59-b78a-4cf6-91a5-22392bc6773d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:54.666462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d39a59-b78a-4cf6-91a5-22392bc6773d admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:54.667519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d39a59-b78a-4cf6-91a5-22392bc6773d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:54.669677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d39a59-b78a-4cf6-91a5-22392bc6773d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:54.796916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 962/1953] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:32:54 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:32:54.803855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80764b1d-a2cb-4afa-acc6-e17fc7baa075 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:54.819232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80764b1d-a2cb-4afa-acc6-e17fc7baa075 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:54.828018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328724b-7ad2-4916-838e-0b13f5153e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:54.829279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328724b-7ad2-4916-838e-0b13f5153e28 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:54.829885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328724b-7ad2-4916-838e-0b13f5153e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:54.830453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328724b-7ad2-4916-838e-0b13f5153e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:54.831649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328724b-7ad2-4916-838e-0b13f5153e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:54.834781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328724b-7ad2-4916-838e-0b13f5153e28 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:54.859614 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 992/1954] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:54.865955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d39a59-b78a-4cf6-91a5-22392bc6773d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:54.885798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d39a59-b78a-4cf6-91a5-22392bc6773d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:54.894177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc194ee-6b41-432e-8a35-40fbd82369f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:54.895106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc194ee-6b41-432e-8a35-40fbd82369f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:54.895590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc194ee-6b41-432e-8a35-40fbd82369f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:54.896157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc194ee-6b41-432e-8a35-40fbd82369f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:54.897001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc194ee-6b41-432e-8a35-40fbd82369f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:54.900766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc194ee-6b41-432e-8a35-40fbd82369f0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:54.920574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 963/1955] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:32:54.922371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328724b-7ad2-4916-838e-0b13f5153e28 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:54.941836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328724b-7ad2-4916-838e-0b13f5153e28 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:54.953517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d74ee-76ed-42cf-9664-169c8301d230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:54.954962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d74ee-76ed-42cf-9664-169c8301d230 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:54.955550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d74ee-76ed-42cf-9664-169c8301d230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:54.956123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d74ee-76ed-42cf-9664-169c8301d230 admin admin] PATH_INFO: `/v3/projects/9e1efaa838174db3a33c5b773505b1bc/users/b92984b9d5b74bde81231b03d392e3f2/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:54.986714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d74ee-76ed-42cf-9664-169c8301d230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1efaa838174db3a33c5b773505b1bc, user_id=b92984b9d5b74bde81231b03d392e3f2, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:54.992450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d74ee-76ed-42cf-9664-169c8301d230 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:55.115939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 993/1956] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:32:54 2022] PUT /identity/v3/projects/9e1efaa838174db3a33c5b773505b1bc/users/b92984b9d5b74bde81231b03d392e3f2/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:32:55.128238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf956b0-78e1-4398-9933-e7cd2f153d43 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:55.129071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf956b0-78e1-4398-9933-e7cd2f153d43 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:55.129276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf956b0-78e1-4398-9933-e7cd2f153d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:55.200431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-1bf956b0-78e1-4398-9933-e7cd2f153d43 None None] MFA Rules not processed for user `b92984b9d5b74bde81231b03d392e3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:55.200431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1bf956b0-78e1-4398-9933-e7cd2f153d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:55.360018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17396a0-8535-498b-b94e-a92b6c08b468 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:55.360587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17396a0-8535-498b-b94e-a92b6c08b468 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:55.360774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17396a0-8535-498b-b94e-a92b6c08b468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:55.399495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 964/1957] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:32:55.411650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf956b0-78e1-4398-9933-e7cd2f153d43 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:55.413137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1bf956b0-78e1-4398-9933-e7cd2f153d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:55.422512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a17396a0-8535-498b-b94e-a92b6c08b468 None None] MFA Rules not processed for user `0cf6efaa09a440d0a17564c9d48196f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:32:55.427877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a17396a0-8535-498b-b94e-a92b6c08b468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:55.651013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 994/1958] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:32:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:32:55.660566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17396a0-8535-498b-b94e-a92b6c08b468 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:55.664347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a17396a0-8535-498b-b94e-a92b6c08b468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:55.667871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf956b0-78e1-4398-9933-e7cd2f153d43 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:55.680077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9bc01-fd15-4371-9a10-9e85786da3f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ac781cf2d84d95a789724e74e12acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:55.680398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:55.680745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9bc01-fd15-4371-9a10-9e85786da3f8 service glance] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:55.681032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9bc01-fd15-4371-9a10-9e85786da3f8 service glance] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:55.681342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9bc01-fd15-4371-9a10-9e85786da3f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:55.700677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9bc01-fd15-4371-9a10-9e85786da3f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:55.703441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9bc01-fd15-4371-9a10-9e85786da3f8 service glance] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:55.707890 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:32:55.735304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 965/1959] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:32:55.801038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17396a0-8535-498b-b94e-a92b6c08b468 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:55.809055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:55.810101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:55.810615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:55.811071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:55.816815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:55.917279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:55.920366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:55.959017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 995/1960] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:32:59.478717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9bc01-fd15-4371-9a10-9e85786da3f8 service glance] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:32:59.480990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-43e9bc01-fd15-4371-9a10-9e85786da3f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:32:59.499449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9bc01-fd15-4371-9a10-9e85786da3f8 service glance] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:32:59.505762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a52cbd-afc9-44d0-809b-1643ee333bf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:32:59.506392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a52cbd-afc9-44d0-809b-1643ee333bf3 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:32:59.506730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a52cbd-afc9-44d0-809b-1643ee333bf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:32:59.507881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a52cbd-afc9-44d0-809b-1643ee333bf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:32:59.528017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a52cbd-afc9-44d0-809b-1643ee333bf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:32:59.530942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a52cbd-afc9-44d0-809b-1643ee333bf3 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:32:59.552022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 966/1961] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:32:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:06.586830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=46 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:09.570476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=7 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:26.464071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:26.465957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:26.569029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986a050-92f1-475d-bd9b-665a2d60d94c service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:26.574488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:26.575172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:26.575446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:26.575798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] PATH_INFO: `/v3/users/a48caf30ea4940998a692c71230378ed` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:26.577407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48caf30ea4940998a692c71230378ed)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:26.579353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:26.817400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48caf30ea4940998a692c71230378ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:26.873240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48caf30ea4940998a692c71230378ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:26.898594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48caf30ea4940998a692c71230378ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:26.899563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48caf30ea4940998a692c71230378ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:26.913533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48caf30ea4940998a692c71230378ed'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:26.924182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 996/1962] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:26 2022] DELETE /identity/v3/users/a48caf30ea4940998a692c71230378ed => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:33:27.386383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a52cbd-afc9-44d0-809b-1643ee333bf3 service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:27.404337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a52cbd-afc9-44d0-809b-1643ee333bf3 service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:27.505454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:27.506101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:27.506385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:27.506655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] PATH_INFO: `/v3/projects/a66dd577cfe8414bb4bc31c42db72e90` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:27.507833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66dd577cfe8414bb4bc31c42db72e90)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:27.509851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:27.755240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66dd577cfe8414bb4bc31c42db72e90', 'request_id': 'req-f4ee42d7-de0e-4787-bf74-48add4a77eb0'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:27.764486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66dd577cfe8414bb4bc31c42db72e90', 'request_id': 'req-f4ee42d7-de0e-4787-bf74-48add4a77eb0'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:27.948358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66dd577cfe8414bb4bc31c42db72e90', 'request_id': 'req-f4ee42d7-de0e-4787-bf74-48add4a77eb0'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:27.973124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] The token cache is being invalidate because project a66dd577cfe8414bb4bc31c42db72e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:33:27.973808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:27.974965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:27.979440 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 967/1963] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:33:27 2022] DELETE /identity/v3/projects/a66dd577cfe8414bb4bc31c42db72e90 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:27.984879 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:27.986825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:28.004346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d88d0f-1f6f-4feb-a330-81bc7f8d9eea admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:28.074726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:28.075440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:28.076461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:28.076781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] PATH_INFO: `/v3/users/0cf6efaa09a440d0a17564c9d48196f3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:28.078267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf6efaa09a440d0a17564c9d48196f3)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:28.080450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:28.105767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:28.537527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf6efaa09a440d0a17564c9d48196f3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:28.667179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf6efaa09a440d0a17564c9d48196f3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:28.686541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf6efaa09a440d0a17564c9d48196f3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:28.687413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf6efaa09a440d0a17564c9d48196f3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:28.697773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b8018ea3-e871-491d-8ed9-1218da0b611d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf6efaa09a440d0a17564c9d48196f3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:28.706066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 997/1964] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:27 2022] DELETE /identity/v3/users/0cf6efaa09a440d0a17564c9d48196f3 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:29.091641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:29.093231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:29.122811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee42d7-de0e-4787-bf74-48add4a77eb0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:29.212210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:29.213305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:29.213802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:29.214298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] PATH_INFO: `/v3/projects/88cbb5cd9b664838a3c70e4017c54ba9` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:29.217008 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88cbb5cd9b664838a3c70e4017c54ba9)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:29.219981 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:29.336186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:29.357831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88cbb5cd9b664838a3c70e4017c54ba9', 'request_id': 'req-8f2579de-886e-4d0a-bc42-292d48162490'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:29.364620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88cbb5cd9b664838a3c70e4017c54ba9', 'request_id': 'req-8f2579de-886e-4d0a-bc42-292d48162490'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:29.530632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88cbb5cd9b664838a3c70e4017c54ba9', 'request_id': 'req-8f2579de-886e-4d0a-bc42-292d48162490'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:29.553009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] The token cache is being invalidate because project 88cbb5cd9b664838a3c70e4017c54ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:33:29.553503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:29.554628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f2579de-886e-4d0a-bc42-292d48162490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:29.563329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 968/1965] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:33:29 2022] DELETE /identity/v3/projects/88cbb5cd9b664838a3c70e4017c54ba9 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:29.565280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640713d-1d13-496f-a58f-6136d2973c8d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:29.566130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640713d-1d13-496f-a58f-6136d2973c8d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:29.566130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640713d-1d13-496f-a58f-6136d2973c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:29.591289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a640713d-1d13-496f-a58f-6136d2973c8d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:29.593441 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a640713d-1d13-496f-a58f-6136d2973c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:29.725785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 998/1966] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:29.742266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489e4bf-5801-4e81-9d28-eac40eacfdef None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:29.742859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489e4bf-5801-4e81-9d28-eac40eacfdef None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:29.743366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489e4bf-5801-4e81-9d28-eac40eacfdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:29.778342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c489e4bf-5801-4e81-9d28-eac40eacfdef None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:29.782724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c489e4bf-5801-4e81-9d28-eac40eacfdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:29.803156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 969/1967] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:29.810694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640713d-1d13-496f-a58f-6136d2973c8d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:29.825967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640713d-1d13-496f-a58f-6136d2973c8d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:29.831411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca74f8-cc5f-45e8-b00a-8b7865d176cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:29.832136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca74f8-cc5f-45e8-b00a-8b7865d176cd admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:29.832527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca74f8-cc5f-45e8-b00a-8b7865d176cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:29.832948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca74f8-cc5f-45e8-b00a-8b7865d176cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:29.833807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca74f8-cc5f-45e8-b00a-8b7865d176cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:33:29.834204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca74f8-cc5f-45e8-b00a-8b7865d176cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:29.836472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca74f8-cc5f-45e8-b00a-8b7865d176cd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:29.859222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 999/1968] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:33:29.865516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489e4bf-5801-4e81-9d28-eac40eacfdef None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:29.879575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489e4bf-5801-4e81-9d28-eac40eacfdef None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:29.895233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f97f8c-c08e-4d8a-9cbf-53e2bfa3a266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:29.896375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f97f8c-c08e-4d8a-9cbf-53e2bfa3a266 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:29.896985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f97f8c-c08e-4d8a-9cbf-53e2bfa3a266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:29.897526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f97f8c-c08e-4d8a-9cbf-53e2bfa3a266 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:29.898740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f97f8c-c08e-4d8a-9cbf-53e2bfa3a266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:29.901324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f97f8c-c08e-4d8a-9cbf-53e2bfa3a266 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:30.150739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 970/1969] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:29 2022] POST /identity/v3/projects => generated 369 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:33:30.155046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca74f8-cc5f-45e8-b00a-8b7865d176cd admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:30.177271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca74f8-cc5f-45e8-b00a-8b7865d176cd admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:30.285470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276066ee-fd79-4ab6-a6fb-ce1aeebafd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:30.286090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276066ee-fd79-4ab6-a6fb-ce1aeebafd7a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:30.286366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276066ee-fd79-4ab6-a6fb-ce1aeebafd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:30.286636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276066ee-fd79-4ab6-a6fb-ce1aeebafd7a admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:30.287282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276066ee-fd79-4ab6-a6fb-ce1aeebafd7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:30.289270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276066ee-fd79-4ab6-a6fb-ce1aeebafd7a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:30.504088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1000/1970] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:30 2022] POST /identity/v3/users => generated 285 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:30.510824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f97f8c-c08e-4d8a-9cbf-53e2bfa3a266 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:30.532894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f97f8c-c08e-4d8a-9cbf-53e2bfa3a266 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:30.538347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63df161-5f16-4cc8-94e3-94f5895b0d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:30.539480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63df161-5f16-4cc8-94e3-94f5895b0d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:30.539907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63df161-5f16-4cc8-94e3-94f5895b0d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:30.540429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63df161-5f16-4cc8-94e3-94f5895b0d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:30.541740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63df161-5f16-4cc8-94e3-94f5895b0d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:30.544990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63df161-5f16-4cc8-94e3-94f5895b0d26 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:30.566071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 971/1971] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:30.586896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276066ee-fd79-4ab6-a6fb-ce1aeebafd7a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:30.588348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276066ee-fd79-4ab6-a6fb-ce1aeebafd7a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:30.596040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa885981-7ffc-4cab-8c29-6997609b386e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:30.597166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa885981-7ffc-4cab-8c29-6997609b386e admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:30.597668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa885981-7ffc-4cab-8c29-6997609b386e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:30.598324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa885981-7ffc-4cab-8c29-6997609b386e admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:30.599585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa885981-7ffc-4cab-8c29-6997609b386e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:30.602754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa885981-7ffc-4cab-8c29-6997609b386e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:30.632990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1001/1972] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:30.634406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63df161-5f16-4cc8-94e3-94f5895b0d26 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:30.655631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63df161-5f16-4cc8-94e3-94f5895b0d26 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:30.655857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8546d-5413-43d4-8871-6ecf7e746d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:30.656983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8546d-5413-43d4-8871-6ecf7e746d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:30.657480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8546d-5413-43d4-8871-6ecf7e746d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:30.657955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8546d-5413-43d4-8871-6ecf7e746d4a admin admin] PATH_INFO: `/v3/projects/b862f93ad7b5482b8f73f0eeab9e3ad1/users/cf5f20c741824cb4b781312bca396485/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:30.689751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8546d-5413-43d4-8871-6ecf7e746d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b862f93ad7b5482b8f73f0eeab9e3ad1, user_id=cf5f20c741824cb4b781312bca396485, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:30.695438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8546d-5413-43d4-8871-6ecf7e746d4a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:30.898448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 972/1973] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:30 2022] PUT /identity/v3/projects/b862f93ad7b5482b8f73f0eeab9e3ad1/users/cf5f20c741824cb4b781312bca396485/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:30.912027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395418f-4d42-4d44-8f05-089ae4d98654 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:30.912379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395418f-4d42-4d44-8f05-089ae4d98654 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:30.912656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395418f-4d42-4d44-8f05-089ae4d98654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:30.922919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:30.955978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5395418f-4d42-4d44-8f05-089ae4d98654 None None] MFA Rules not processed for user `cf5f20c741824cb4b781312bca396485`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:30.957970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5395418f-4d42-4d44-8f05-089ae4d98654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:31.075256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1002/1974] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:31.089436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8546d-5413-43d4-8871-6ecf7e746d4a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:31.097281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c9e8546d-5413-43d4-8871-6ecf7e746d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:31.205520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8546d-5413-43d4-8871-6ecf7e746d4a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:31.210689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b2323-a391-44ad-a2b1-e156f6859199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:31.211712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b2323-a391-44ad-a2b1-e156f6859199 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:31.212070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b2323-a391-44ad-a2b1-e156f6859199 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:31.212343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b2323-a391-44ad-a2b1-e156f6859199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:31.228940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b2323-a391-44ad-a2b1-e156f6859199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:31.230418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b2323-a391-44ad-a2b1-e156f6859199 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:31.241126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 973/1975] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:31.268691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395418f-4d42-4d44-8f05-089ae4d98654 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:31.343659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395418f-4d42-4d44-8f05-089ae4d98654 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:31.349941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:31.350722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:31.351043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:31.351410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] PATH_INFO: `/v3/users/cf5f20c741824cb4b781312bca396485` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:31.352863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5f20c741824cb4b781312bca396485)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:31.354981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:31.782682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5f20c741824cb4b781312bca396485'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:31.931233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5f20c741824cb4b781312bca396485'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:31.965015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5f20c741824cb4b781312bca396485'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:31.966004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5f20c741824cb4b781312bca396485'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:31.979329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5f20c741824cb4b781312bca396485'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:31.990452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1003/1976] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:31 2022] DELETE /identity/v3/users/cf5f20c741824cb4b781312bca396485 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:31.998174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525fb42-9dc4-40b5-beac-77687381b189 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:31.998959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525fb42-9dc4-40b5-beac-77687381b189 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:31.998959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525fb42-9dc4-40b5-beac-77687381b189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:32.034002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b525fb42-9dc4-40b5-beac-77687381b189 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:32.038965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b525fb42-9dc4-40b5-beac-77687381b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:32.156755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:32.173292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 974/1977] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:32.183058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:32.188870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:32.215654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:32.338422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837ab27-8190-42e5-9211-2294ea9ae725 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:32.345619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2ca9d-7e60-4408-a951-7f79731f819a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:32.346711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2ca9d-7e60-4408-a951-7f79731f819a service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:32.347207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2ca9d-7e60-4408-a951-7f79731f819a service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:32.347824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2ca9d-7e60-4408-a951-7f79731f819a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:32.364069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2ca9d-7e60-4408-a951-7f79731f819a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:32.365587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2ca9d-7e60-4408-a951-7f79731f819a service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:32.377784 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1004/1978] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:33.538874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525fb42-9dc4-40b5-beac-77687381b189 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:33.542664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b525fb42-9dc4-40b5-beac-77687381b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:33.565067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525fb42-9dc4-40b5-beac-77687381b189 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:33.578923 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:33.579871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:33.580286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:33.580715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] PATH_INFO: `/v3/projects/b862f93ad7b5482b8f73f0eeab9e3ad1` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:33.582754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b862f93ad7b5482b8f73f0eeab9e3ad1)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:33.586555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:33.838948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b862f93ad7b5482b8f73f0eeab9e3ad1', 'request_id': 'req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:33.874948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b862f93ad7b5482b8f73f0eeab9e3ad1', 'request_id': 'req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:34.045683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b862f93ad7b5482b8f73f0eeab9e3ad1', 'request_id': 'req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:34.070875 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] The token cache is being invalidate because project b862f93ad7b5482b8f73f0eeab9e3ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:33:34.071552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:34.072520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f2a65de7-e7c2-414c-8f4b-dad56a8997f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:34.081469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0219ee-be56-4172-a749-e0b609a0ffdf None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:34.084265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0219ee-be56-4172-a749-e0b609a0ffdf None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:34.091878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0219ee-be56-4172-a749-e0b609a0ffdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:34.092078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 975/1979] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:33:33 2022] DELETE /identity/v3/projects/b862f93ad7b5482b8f73f0eeab9e3ad1 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:34.125427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0f0219ee-be56-4172-a749-e0b609a0ffdf None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:34.131312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0f0219ee-be56-4172-a749-e0b609a0ffdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:34.243782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1005/1980] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:34.260253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7220b-f112-427c-8d03-4895ef170637 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:34.260729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7220b-f112-427c-8d03-4895ef170637 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:34.261264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7220b-f112-427c-8d03-4895ef170637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:34.292079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-7fa7220b-f112-427c-8d03-4895ef170637 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:34.294341 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7fa7220b-f112-427c-8d03-4895ef170637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:34.307596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 976/1981] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:34.315269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0219ee-be56-4172-a749-e0b609a0ffdf None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:34.328031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0219ee-be56-4172-a749-e0b609a0ffdf None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:34.332894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32c06f-a028-483d-bf0e-e2326ac8e2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:34.333557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32c06f-a028-483d-bf0e-e2326ac8e2ee admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:34.333889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32c06f-a028-483d-bf0e-e2326ac8e2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:34.334238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32c06f-a028-483d-bf0e-e2326ac8e2ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:34.335028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32c06f-a028-483d-bf0e-e2326ac8e2ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:33:34.335366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32c06f-a028-483d-bf0e-e2326ac8e2ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:34.337060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32c06f-a028-483d-bf0e-e2326ac8e2ee admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:34.357210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1006/1982] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:33:34.363568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7220b-f112-427c-8d03-4895ef170637 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:34.377440 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7220b-f112-427c-8d03-4895ef170637 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:34.390368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ec07d-318a-4bb9-8340-cbd6130565b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:34.390368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ec07d-318a-4bb9-8340-cbd6130565b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:34.390368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ec07d-318a-4bb9-8340-cbd6130565b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:34.390368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ec07d-318a-4bb9-8340-cbd6130565b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:34.391035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ec07d-318a-4bb9-8340-cbd6130565b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:34.391035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ec07d-318a-4bb9-8340-cbd6130565b8 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:34.633131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 977/1983] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:34 2022] POST /identity/v3/projects => generated 375 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:34.638746 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32c06f-a028-483d-bf0e-e2326ac8e2ee admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:34.664408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32c06f-a028-483d-bf0e-e2326ac8e2ee admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:34.781060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079899e-e1ea-4e75-af86-d470f7a2931a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:34.782127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079899e-e1ea-4e75-af86-d470f7a2931a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:34.782578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079899e-e1ea-4e75-af86-d470f7a2931a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:34.783060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079899e-e1ea-4e75-af86-d470f7a2931a admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:34.784223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079899e-e1ea-4e75-af86-d470f7a2931a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:34.788260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079899e-e1ea-4e75-af86-d470f7a2931a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:34.925175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1007/1984] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:34 2022] POST /identity/v3/users => generated 288 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:34.927235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ec07d-318a-4bb9-8340-cbd6130565b8 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:34.960878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ec07d-318a-4bb9-8340-cbd6130565b8 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:35.003508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696ad9a-3a63-4b40-ac83-ea570178a6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:35.004133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696ad9a-3a63-4b40-ac83-ea570178a6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:35.004517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696ad9a-3a63-4b40-ac83-ea570178a6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:35.004774 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696ad9a-3a63-4b40-ac83-ea570178a6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:35.005628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696ad9a-3a63-4b40-ac83-ea570178a6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:35.007346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696ad9a-3a63-4b40-ac83-ea570178a6e0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:35.026949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 978/1985] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:35.036024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079899e-e1ea-4e75-af86-d470f7a2931a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:35.052621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079899e-e1ea-4e75-af86-d470f7a2931a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:35.059208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23e671-feca-487e-8474-f8f7b290d758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:35.060711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23e671-feca-487e-8474-f8f7b290d758 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:35.061157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23e671-feca-487e-8474-f8f7b290d758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:35.061553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23e671-feca-487e-8474-f8f7b290d758 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:35.062357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de23e671-feca-487e-8474-f8f7b290d758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:35.064136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de23e671-feca-487e-8474-f8f7b290d758 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:35.079304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1008/1986] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:35.086396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696ad9a-3a63-4b40-ac83-ea570178a6e0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:35.117174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696ad9a-3a63-4b40-ac83-ea570178a6e0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:35.125219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079598e-1dcb-4581-9e54-1d32f0e331f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:35.126472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079598e-1dcb-4581-9e54-1d32f0e331f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:35.126936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079598e-1dcb-4581-9e54-1d32f0e331f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:35.127408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079598e-1dcb-4581-9e54-1d32f0e331f0 admin admin] PATH_INFO: `/v3/projects/786b0b5cd1274e269b7dbeecc85c043d/users/6b2936c10531445994423d5fcbe69220/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:35.175000 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079598e-1dcb-4581-9e54-1d32f0e331f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=786b0b5cd1274e269b7dbeecc85c043d, user_id=6b2936c10531445994423d5fcbe69220, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:35.177188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079598e-1dcb-4581-9e54-1d32f0e331f0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:35.307114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 979/1987] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:35 2022] PUT /identity/v3/projects/786b0b5cd1274e269b7dbeecc85c043d/users/6b2936c10531445994423d5fcbe69220/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:35.327470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb2e1-e2bc-4195-8678-9283f1742c50 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:35.342675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb2e1-e2bc-4195-8678-9283f1742c50 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:35.342675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb2e1-e2bc-4195-8678-9283f1742c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:35.419417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7feeb2e1-e2bc-4195-8678-9283f1742c50 None None] MFA Rules not processed for user `6b2936c10531445994423d5fcbe69220`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:35.421396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7feeb2e1-e2bc-4195-8678-9283f1742c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:35.453263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:35.613444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1009/1988] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:35.615849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079598e-1dcb-4581-9e54-1d32f0e331f0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:35.780190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079598e-1dcb-4581-9e54-1d32f0e331f0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:35.805859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a801-795b-4df7-a9f3-2d34346ae682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:35.807003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a801-795b-4df7-a9f3-2d34346ae682 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:35.813396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a801-795b-4df7-a9f3-2d34346ae682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:35.813396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a801-795b-4df7-a9f3-2d34346ae682 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:35.813396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a801-795b-4df7-a9f3-2d34346ae682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:35.814031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a801-795b-4df7-a9f3-2d34346ae682 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:36.036904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 980/1989] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:35 2022] POST /identity/v3/projects => generated 375 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:33:36.041095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeb2e1-e2bc-4195-8678-9283f1742c50 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:36.054505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeb2e1-e2bc-4195-8678-9283f1742c50 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:36.150267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4057ec4-ff60-4580-ac74-000ef27d0336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:36.151116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4057ec4-ff60-4580-ac74-000ef27d0336 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:36.151397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4057ec4-ff60-4580-ac74-000ef27d0336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:36.151776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4057ec4-ff60-4580-ac74-000ef27d0336 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:36.161286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4057ec4-ff60-4580-ac74-000ef27d0336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:36.161286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4057ec4-ff60-4580-ac74-000ef27d0336 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:36.316480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1010/1990] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:36 2022] POST /identity/v3/users => generated 294 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:36.320494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a801-795b-4df7-a9f3-2d34346ae682 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:36.335093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:36.342416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a801-795b-4df7-a9f3-2d34346ae682 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:36.351498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f500b6-e11c-40ad-adf7-35b838edc765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:36.352502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f500b6-e11c-40ad-adf7-35b838edc765 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:36.353126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f500b6-e11c-40ad-adf7-35b838edc765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:36.353837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f500b6-e11c-40ad-adf7-35b838edc765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:36.354782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f500b6-e11c-40ad-adf7-35b838edc765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:36.357312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f500b6-e11c-40ad-adf7-35b838edc765 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:36.373337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 981/1991] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:36.380015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4057ec4-ff60-4580-ac74-000ef27d0336 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:36.399073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4057ec4-ff60-4580-ac74-000ef27d0336 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:36.409051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabcc47-9c47-42be-9139-e0d8ec397c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:36.410283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabcc47-9c47-42be-9139-e0d8ec397c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:36.410537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabcc47-9c47-42be-9139-e0d8ec397c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:36.411403 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabcc47-9c47-42be-9139-e0d8ec397c07 admin admin] PATH_INFO: `/v3/projects/835432b860ec471db3019169348ee742/users/c66f476eebd84c1dad21a6866c625aed/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:36.439620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabcc47-9c47-42be-9139-e0d8ec397c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835432b860ec471db3019169348ee742, user_id=c66f476eebd84c1dad21a6866c625aed, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:36.446198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabcc47-9c47-42be-9139-e0d8ec397c07 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:36.511665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f500b6-e11c-40ad-adf7-35b838edc765 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:36.516606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-24f500b6-e11c-40ad-adf7-35b838edc765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:36.541980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f500b6-e11c-40ad-adf7-35b838edc765 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:36.548236 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:36.552073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:36.552073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:36.552073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] PATH_INFO: `/v3/users/77cef1a9e6a241ac8401b88f4a0f69fe` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:36.552073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77cef1a9e6a241ac8401b88f4a0f69fe)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:36.554456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:36.602727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1011/1992] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:36 2022] PUT /identity/v3/projects/835432b860ec471db3019169348ee742/users/c66f476eebd84c1dad21a6866c625aed/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:36.609602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabcc47-9c47-42be-9139-e0d8ec397c07 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:36.739569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabcc47-9c47-42be-9139-e0d8ec397c07 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:36.747568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2958cae-8ce8-4c41-8cb1-263c517467cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:36.748687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2958cae-8ce8-4c41-8cb1-263c517467cb admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:36.749224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2958cae-8ce8-4c41-8cb1-263c517467cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:36.749661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2958cae-8ce8-4c41-8cb1-263c517467cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:36.750823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2958cae-8ce8-4c41-8cb1-263c517467cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:36.754268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2958cae-8ce8-4c41-8cb1-263c517467cb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:36.771170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1012/1993] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:33:36.777640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2958cae-8ce8-4c41-8cb1-263c517467cb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:36.800426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2958cae-8ce8-4c41-8cb1-263c517467cb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:36.807495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44b131-19cc-4346-93d3-b7cfe68ba17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:36.808589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44b131-19cc-4346-93d3-b7cfe68ba17f admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:36.809096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44b131-19cc-4346-93d3-b7cfe68ba17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:36.810557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44b131-19cc-4346-93d3-b7cfe68ba17f admin admin] PATH_INFO: `/v3/projects/835432b860ec471db3019169348ee742/users/c66f476eebd84c1dad21a6866c625aed/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:36.814040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44b131-19cc-4346-93d3-b7cfe68ba17f admin admin] RBAC: Authorizing `identity:create_grant(project_id=835432b860ec471db3019169348ee742, user_id=c66f476eebd84c1dad21a6866c625aed, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:36.817710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b44b131-19cc-4346-93d3-b7cfe68ba17f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:36.829066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:36.834574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1013/1994] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:36 2022] PUT /identity/v3/projects/835432b860ec471db3019169348ee742/users/c66f476eebd84c1dad21a6866c625aed/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:36.853352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7212693-6f3d-4680-a3cf-7de2e5499bf1 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:36.853782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7212693-6f3d-4680-a3cf-7de2e5499bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:36.854143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7212693-6f3d-4680-a3cf-7de2e5499bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:36.907536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d7212693-6f3d-4680-a3cf-7de2e5499bf1 None None] MFA Rules not processed for user `c66f476eebd84c1dad21a6866c625aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:36.909729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d7212693-6f3d-4680-a3cf-7de2e5499bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:36.947879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77cef1a9e6a241ac8401b88f4a0f69fe'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:36.948690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77cef1a9e6a241ac8401b88f4a0f69fe'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:36.949830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77cef1a9e6a241ac8401b88f4a0f69fe'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:36.966037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77cef1a9e6a241ac8401b88f4a0f69fe'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:36.990192 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77cef1a9e6a241ac8401b88f4a0f69fe'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:37.013476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1014/1995] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:37.033315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7212693-6f3d-4680-a3cf-7de2e5499bf1 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:37.033315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d7212693-6f3d-4680-a3cf-7de2e5499bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:37.107875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7212693-6f3d-4680-a3cf-7de2e5499bf1 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:37.111892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a84362-2df9-4665-8a31-de88244e015a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:37.112515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a84362-2df9-4665-8a31-de88244e015a service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:37.112789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a84362-2df9-4665-8a31-de88244e015a service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:37.113259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a84362-2df9-4665-8a31-de88244e015a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:37.115255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-61a84362-2df9-4665-8a31-de88244e015a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:37.213657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 982/1996] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:36 2022] DELETE /identity/v3/users/77cef1a9e6a241ac8401b88f4a0f69fe => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:37.227685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a84362-2df9-4665-8a31-de88244e015a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:37.228369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a84362-2df9-4665-8a31-de88244e015a service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:37.242355 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1015/1997] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:33:37.786836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:37.795351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:37.938228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b445b9e-7a8a-4f51-894f-f2a8bcb98b94 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:37.952038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:37.953248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:37.953864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:37.954645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] PATH_INFO: `/v3/projects/901780013b03485a83261796f69e1b01` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:37.964303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=901780013b03485a83261796f69e1b01)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:37.978768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:38.209865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901780013b03485a83261796f69e1b01', 'request_id': 'req-2857f752-601f-4773-afaf-ddd0674e9c4a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:38.218363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901780013b03485a83261796f69e1b01', 'request_id': 'req-2857f752-601f-4773-afaf-ddd0674e9c4a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:38.322327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901780013b03485a83261796f69e1b01', 'request_id': 'req-2857f752-601f-4773-afaf-ddd0674e9c4a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:38.325696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:38.343590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] The token cache is being invalidate because project 901780013b03485a83261796f69e1b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:33:38.344173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:38.345030 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2857f752-601f-4773-afaf-ddd0674e9c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:38.346616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 983/1998] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:33:37 2022] DELETE /identity/v3/projects/901780013b03485a83261796f69e1b01 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:38.354854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af51bf-70e8-4dbe-bb90-06620c7db839 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:38.355574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af51bf-70e8-4dbe-bb90-06620c7db839 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:38.356205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af51bf-70e8-4dbe-bb90-06620c7db839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:38.387400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e9af51bf-70e8-4dbe-bb90-06620c7db839 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:38.390026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e9af51bf-70e8-4dbe-bb90-06620c7db839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:38.451689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:38.496462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1016/1999] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:38.506353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef973067-1eb7-455a-82e7-ff059badd7eb None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:38.506831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef973067-1eb7-455a-82e7-ff059badd7eb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:38.507626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef973067-1eb7-455a-82e7-ff059badd7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:38.536322 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ef973067-1eb7-455a-82e7-ff059badd7eb None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:38.542542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ef973067-1eb7-455a-82e7-ff059badd7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:38.566283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 984/2000] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:38.578608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af51bf-70e8-4dbe-bb90-06620c7db839 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:38.600516 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af51bf-70e8-4dbe-bb90-06620c7db839 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:38.609852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7b58a-9ab0-457e-8267-31f262b36452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:38.611028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7b58a-9ab0-457e-8267-31f262b36452 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:38.611393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7b58a-9ab0-457e-8267-31f262b36452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:38.611942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7b58a-9ab0-457e-8267-31f262b36452 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:38.613211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7b58a-9ab0-457e-8267-31f262b36452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:33:38.613756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7b58a-9ab0-457e-8267-31f262b36452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:38.616907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7b58a-9ab0-457e-8267-31f262b36452 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:38.637414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1017/2001] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:33:38.644852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef973067-1eb7-455a-82e7-ff059badd7eb None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:38.661919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef973067-1eb7-455a-82e7-ff059badd7eb None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:38.669902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e62d61-9bc0-406d-842d-e9ab0c672b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:38.670541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e62d61-9bc0-406d-842d-e9ab0c672b27 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:38.670815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e62d61-9bc0-406d-842d-e9ab0c672b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:38.671087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e62d61-9bc0-406d-842d-e9ab0c672b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:38.671739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e62d61-9bc0-406d-842d-e9ab0c672b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:38.673692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e62d61-9bc0-406d-842d-e9ab0c672b27 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:38.826852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 985/2002] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:38 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:38.828130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7b58a-9ab0-457e-8267-31f262b36452 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:38.839432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7b58a-9ab0-457e-8267-31f262b36452 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:38.947073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a99a8-8b22-44ce-8432-d413cf9b516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:38.947897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a99a8-8b22-44ce-8432-d413cf9b516f admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:38.948266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a99a8-8b22-44ce-8432-d413cf9b516f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:38.948637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a99a8-8b22-44ce-8432-d413cf9b516f admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:38.949308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a99a8-8b22-44ce-8432-d413cf9b516f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:38.951637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a99a8-8b22-44ce-8432-d413cf9b516f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:39.146812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1018/2003] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:38 2022] POST /identity/v3/users => generated 291 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:39.151432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e62d61-9bc0-406d-842d-e9ab0c672b27 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:39.162344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e62d61-9bc0-406d-842d-e9ab0c672b27 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:39.168519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3baa3f-1e54-4381-8cfa-f8a79b197724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:39.169669 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3baa3f-1e54-4381-8cfa-f8a79b197724 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:39.170028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3baa3f-1e54-4381-8cfa-f8a79b197724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:39.170218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3baa3f-1e54-4381-8cfa-f8a79b197724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:39.170878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3baa3f-1e54-4381-8cfa-f8a79b197724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:39.172534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3baa3f-1e54-4381-8cfa-f8a79b197724 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:39.185356 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 986/2004] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:39.190205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a99a8-8b22-44ce-8432-d413cf9b516f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:39.209847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a99a8-8b22-44ce-8432-d413cf9b516f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:39.216938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662da1f-d286-48d6-8d71-e51f04621c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:39.217438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662da1f-d286-48d6-8d71-e51f04621c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:39.217715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662da1f-d286-48d6-8d71-e51f04621c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:39.217984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662da1f-d286-48d6-8d71-e51f04621c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:39.218625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662da1f-d286-48d6-8d71-e51f04621c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:39.220225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662da1f-d286-48d6-8d71-e51f04621c6a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:39.236096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1019/2005] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:39.243271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3baa3f-1e54-4381-8cfa-f8a79b197724 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:39.261103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3baa3f-1e54-4381-8cfa-f8a79b197724 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:39.267310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730a487-0e56-45b6-955d-00038af7762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:39.267985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730a487-0e56-45b6-955d-00038af7762f admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:39.268260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730a487-0e56-45b6-955d-00038af7762f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:39.268596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730a487-0e56-45b6-955d-00038af7762f admin admin] PATH_INFO: `/v3/projects/d92b23b5037846c18c874be807df3e39/users/242a0be3cc7c49a39703f6ceabc36542/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:39.295660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730a487-0e56-45b6-955d-00038af7762f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92b23b5037846c18c874be807df3e39, user_id=242a0be3cc7c49a39703f6ceabc36542, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:39.301433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730a487-0e56-45b6-955d-00038af7762f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:39.461619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 987/2006] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:39 2022] PUT /identity/v3/projects/d92b23b5037846c18c874be807df3e39/users/242a0be3cc7c49a39703f6ceabc36542/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:39.472654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a5939-61f0-4a8a-9de2-5dc419fcc689 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:39.473341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a5939-61f0-4a8a-9de2-5dc419fcc689 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:39.473341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a5939-61f0-4a8a-9de2-5dc419fcc689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:39.504603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f66a5939-61f0-4a8a-9de2-5dc419fcc689 None None] MFA Rules not processed for user `242a0be3cc7c49a39703f6ceabc36542`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:39.505945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f66a5939-61f0-4a8a-9de2-5dc419fcc689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:39.594318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1020/2007] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:39.599410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730a487-0e56-45b6-955d-00038af7762f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:39.711351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730a487-0e56-45b6-955d-00038af7762f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:39.717154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103671cb-4bc8-4d5b-94a4-ec8379d895d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:39.717793 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103671cb-4bc8-4d5b-94a4-ec8379d895d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:39.718065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103671cb-4bc8-4d5b-94a4-ec8379d895d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:39.718335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103671cb-4bc8-4d5b-94a4-ec8379d895d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:39.718981 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103671cb-4bc8-4d5b-94a4-ec8379d895d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:39.721004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103671cb-4bc8-4d5b-94a4-ec8379d895d5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:39.863758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 988/2008] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:39 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:33:39.869549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a5939-61f0-4a8a-9de2-5dc419fcc689 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:39.884607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a5939-61f0-4a8a-9de2-5dc419fcc689 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:39.940199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:39.987028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce03be-f1d4-431e-9bdf-0ff593c3b3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:39.987728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce03be-f1d4-431e-9bdf-0ff593c3b3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:39.988007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce03be-f1d4-431e-9bdf-0ff593c3b3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:39.988280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce03be-f1d4-431e-9bdf-0ff593c3b3cb admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:39.989007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce03be-f1d4-431e-9bdf-0ff593c3b3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:39.991064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce03be-f1d4-431e-9bdf-0ff593c3b3cb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:40.164231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1021/2009] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:39 2022] POST /identity/v3/users => generated 297 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:40.168071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103671cb-4bc8-4d5b-94a4-ec8379d895d5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:40.184425 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103671cb-4bc8-4d5b-94a4-ec8379d895d5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:40.190665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f870610-49fa-4731-9fd8-4f11d7bd7176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:40.191668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f870610-49fa-4731-9fd8-4f11d7bd7176 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:40.192089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f870610-49fa-4731-9fd8-4f11d7bd7176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:40.192579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f870610-49fa-4731-9fd8-4f11d7bd7176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:40.193584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f870610-49fa-4731-9fd8-4f11d7bd7176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:40.195952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f870610-49fa-4731-9fd8-4f11d7bd7176 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:40.214123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 989/2010] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:40.219494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce03be-f1d4-431e-9bdf-0ff593c3b3cb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:40.232772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce03be-f1d4-431e-9bdf-0ff593c3b3cb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:40.239655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7861ee-3f29-4168-a74a-7ceab29420b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:40.240557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7861ee-3f29-4168-a74a-7ceab29420b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:40.240958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7861ee-3f29-4168-a74a-7ceab29420b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:40.241344 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7861ee-3f29-4168-a74a-7ceab29420b1 admin admin] PATH_INFO: `/v3/projects/dd2e3442b7cf4d7cbbd05c7838e880eb/users/e0b9222140dd4dd1835f318ed5a402aa/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:40.268936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7861ee-3f29-4168-a74a-7ceab29420b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd2e3442b7cf4d7cbbd05c7838e880eb, user_id=e0b9222140dd4dd1835f318ed5a402aa, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:40.271712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7861ee-3f29-4168-a74a-7ceab29420b1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:40.566530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1022/2011] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:40 2022] PUT /identity/v3/projects/dd2e3442b7cf4d7cbbd05c7838e880eb/users/e0b9222140dd4dd1835f318ed5a402aa/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:40.576676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f870610-49fa-4731-9fd8-4f11d7bd7176 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:40.590553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f870610-49fa-4731-9fd8-4f11d7bd7176 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:40.668677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7d539-83eb-4f7f-81f9-a92d81afe3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:40.669510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7d539-83eb-4f7f-81f9-a92d81afe3ca admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:40.669996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7d539-83eb-4f7f-81f9-a92d81afe3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:40.670323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7d539-83eb-4f7f-81f9-a92d81afe3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:40.670993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7d539-83eb-4f7f-81f9-a92d81afe3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:40.672738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7d539-83eb-4f7f-81f9-a92d81afe3ca admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:40.697826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 990/2012] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:40.700944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7861ee-3f29-4168-a74a-7ceab29420b1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:40.721467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7861ee-3f29-4168-a74a-7ceab29420b1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:40.731855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b1df-ebca-4150-b438-fb5167dc6872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:40.733303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b1df-ebca-4150-b438-fb5167dc6872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:40.733722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b1df-ebca-4150-b438-fb5167dc6872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:40.734137 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b1df-ebca-4150-b438-fb5167dc6872 admin admin] PATH_INFO: `/v3/projects/dd2e3442b7cf4d7cbbd05c7838e880eb/users/e0b9222140dd4dd1835f318ed5a402aa/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:40.755216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9b1df-ebca-4150-b438-fb5167dc6872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd2e3442b7cf4d7cbbd05c7838e880eb, user_id=e0b9222140dd4dd1835f318ed5a402aa, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:40.758065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9b1df-ebca-4150-b438-fb5167dc6872 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:40.769927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1023/2013] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:40 2022] PUT /identity/v3/projects/dd2e3442b7cf4d7cbbd05c7838e880eb/users/e0b9222140dd4dd1835f318ed5a402aa/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:40.781889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd2b09-f448-4053-a4ae-de187f70b9ef None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:40.782456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd2b09-f448-4053-a4ae-de187f70b9ef None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:40.782741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd2b09-f448-4053-a4ae-de187f70b9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:40.791096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:40.825383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-2bfd2b09-f448-4053-a4ae-de187f70b9ef None None] MFA Rules not processed for user `e0b9222140dd4dd1835f318ed5a402aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:40.826967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2bfd2b09-f448-4053-a4ae-de187f70b9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:40.921791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 991/2014] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:40.954385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b1df-ebca-4150-b438-fb5167dc6872 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:40.956212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c9b9b1df-ebca-4150-b438-fb5167dc6872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:41.032461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b1df-ebca-4150-b438-fb5167dc6872 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:41.036668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af369c4-59ba-402e-82aa-8e505261759d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:41.038423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af369c4-59ba-402e-82aa-8e505261759d service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:41.039483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af369c4-59ba-402e-82aa-8e505261759d service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:41.040564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af369c4-59ba-402e-82aa-8e505261759d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:41.051853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af369c4-59ba-402e-82aa-8e505261759d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:41.053558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af369c4-59ba-402e-82aa-8e505261759d service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:41.067116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1024/2015] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:41.451407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd2b09-f448-4053-a4ae-de187f70b9ef None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:41.464307 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd2b09-f448-4053-a4ae-de187f70b9ef None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:41.470949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785704cb-8690-45bd-a1d2-1c999970c496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:41.472046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785704cb-8690-45bd-a1d2-1c999970c496 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:41.476563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785704cb-8690-45bd-a1d2-1c999970c496 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:41.476563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785704cb-8690-45bd-a1d2-1c999970c496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:41.562442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785704cb-8690-45bd-a1d2-1c999970c496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:41.564089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785704cb-8690-45bd-a1d2-1c999970c496 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:41.576434 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 992/2016] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:42.517078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af369c4-59ba-402e-82aa-8e505261759d service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:42.519501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3af369c4-59ba-402e-82aa-8e505261759d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:42.645375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af369c4-59ba-402e-82aa-8e505261759d service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:42.655147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:42.655147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:42.655374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:42.656232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] PATH_INFO: `/v3/users/b92984b9d5b74bde81231b03d392e3f2` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:42.657496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b92984b9d5b74bde81231b03d392e3f2)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:42.659761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:43.113645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:43.143622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b92984b9d5b74bde81231b03d392e3f2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:43.711437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b92984b9d5b74bde81231b03d392e3f2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:43.733904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b92984b9d5b74bde81231b03d392e3f2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:43.734761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b92984b9d5b74bde81231b03d392e3f2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:43.742382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b92984b9d5b74bde81231b03d392e3f2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:43.749475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1025/2017] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:42 2022] DELETE /identity/v3/users/b92984b9d5b74bde81231b03d392e3f2 => generated 0 bytes in 1234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:43.754718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964974a5-cc77-44ab-a6be-576ea4254971 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:43.755449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964974a5-cc77-44ab-a6be-576ea4254971 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:43.755449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964974a5-cc77-44ab-a6be-576ea4254971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:43.784207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-964974a5-cc77-44ab-a6be-576ea4254971 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:43.786298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-964974a5-cc77-44ab-a6be-576ea4254971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:43.881582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 993/2018] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:43.886908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:43.891825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:43.972407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcc344-339e-4929-9350-4eac956a80b0 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:43.976102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca93f6e-d6fa-4466-b8d8-cdeb384ffe2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:43.977132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca93f6e-d6fa-4466-b8d8-cdeb384ffe2c service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:43.977132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca93f6e-d6fa-4466-b8d8-cdeb384ffe2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:43.977664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca93f6e-d6fa-4466-b8d8-cdeb384ffe2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:43.991864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca93f6e-d6fa-4466-b8d8-cdeb384ffe2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:44.001032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca93f6e-d6fa-4466-b8d8-cdeb384ffe2c service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:44.005960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1026/2019] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:44.568248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964974a5-cc77-44ab-a6be-576ea4254971 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:44.570401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-964974a5-cc77-44ab-a6be-576ea4254971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:44.586827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964974a5-cc77-44ab-a6be-576ea4254971 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:44.595982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:44.597115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:44.603552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:44.603552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] PATH_INFO: `/v3/projects/9e1efaa838174db3a33c5b773505b1bc` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:44.603552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1efaa838174db3a33c5b773505b1bc)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:44.604818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:44.673779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:44.803632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1efaa838174db3a33c5b773505b1bc', 'request_id': 'req-9f49ce24-897f-49d3-8de5-c20e17b04a0f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:44.810555 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1efaa838174db3a33c5b773505b1bc', 'request_id': 'req-9f49ce24-897f-49d3-8de5-c20e17b04a0f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:44.906689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1efaa838174db3a33c5b773505b1bc', 'request_id': 'req-9f49ce24-897f-49d3-8de5-c20e17b04a0f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:44.920027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] The token cache is being invalidate because project 9e1efaa838174db3a33c5b773505b1bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:33:44.920377 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:44.921093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9f49ce24-897f-49d3-8de5-c20e17b04a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:44.922729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 994/2020] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:33:44 2022] DELETE /identity/v3/projects/9e1efaa838174db3a33c5b773505b1bc => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:44.931123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b2bee-6a2e-4957-bd0a-e8c4427123a8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:44.931685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b2bee-6a2e-4957-bd0a-e8c4427123a8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:44.931685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b2bee-6a2e-4957-bd0a-e8c4427123a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:44.957602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e21b2bee-6a2e-4957-bd0a-e8c4427123a8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:44.960283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e21b2bee-6a2e-4957-bd0a-e8c4427123a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:45.044356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1027/2021] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:45.054665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64b635-abc7-4742-8a91-1bad5c5665e0 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:45.055006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64b635-abc7-4742-8a91-1bad5c5665e0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:45.055498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64b635-abc7-4742-8a91-1bad5c5665e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:45.087998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-3a64b635-abc7-4742-8a91-1bad5c5665e0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:45.090154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3a64b635-abc7-4742-8a91-1bad5c5665e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:45.105041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 995/2022] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:45.112270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b2bee-6a2e-4957-bd0a-e8c4427123a8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:45.131688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b2bee-6a2e-4957-bd0a-e8c4427123a8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:45.138323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53803c-01a1-4fe0-ab3f-cda234e40c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:45.141288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53803c-01a1-4fe0-ab3f-cda234e40c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:45.141821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53803c-01a1-4fe0-ab3f-cda234e40c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:45.142352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53803c-01a1-4fe0-ab3f-cda234e40c25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:45.143646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53803c-01a1-4fe0-ab3f-cda234e40c25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:33:45.144315 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53803c-01a1-4fe0-ab3f-cda234e40c25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:45.146560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53803c-01a1-4fe0-ab3f-cda234e40c25 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:45.162318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1028/2023] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:33:45.167390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64b635-abc7-4742-8a91-1bad5c5665e0 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:45.190450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64b635-abc7-4742-8a91-1bad5c5665e0 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:45.198976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b40f43-f6a1-4bd0-8232-99e5bc849c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:45.200230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b40f43-f6a1-4bd0-8232-99e5bc849c79 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:45.200756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b40f43-f6a1-4bd0-8232-99e5bc849c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:45.201528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b40f43-f6a1-4bd0-8232-99e5bc849c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:45.202248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b40f43-f6a1-4bd0-8232-99e5bc849c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:45.204221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b40f43-f6a1-4bd0-8232-99e5bc849c79 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:45.391083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 996/2024] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:45 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:45.395345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53803c-01a1-4fe0-ab3f-cda234e40c25 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:45.408334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53803c-01a1-4fe0-ab3f-cda234e40c25 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:45.481118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba03afb-57dd-4814-a0f9-18bc5ca03ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:45.481923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba03afb-57dd-4814-a0f9-18bc5ca03ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:45.482273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba03afb-57dd-4814-a0f9-18bc5ca03ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:45.482597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba03afb-57dd-4814-a0f9-18bc5ca03ec1 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:45.483269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba03afb-57dd-4814-a0f9-18bc5ca03ec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:45.485350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba03afb-57dd-4814-a0f9-18bc5ca03ec1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:45.541422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:45.598782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1029/2025] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:45 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:45.603739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b40f43-f6a1-4bd0-8232-99e5bc849c79 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:45.616823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b40f43-f6a1-4bd0-8232-99e5bc849c79 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:45.625857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f319f-0ca3-4fc2-9c7d-4d1691d4eddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:45.626497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f319f-0ca3-4fc2-9c7d-4d1691d4eddd admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:45.626777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f319f-0ca3-4fc2-9c7d-4d1691d4eddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:45.627049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f319f-0ca3-4fc2-9c7d-4d1691d4eddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:45.627697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f319f-0ca3-4fc2-9c7d-4d1691d4eddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:45.629388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f319f-0ca3-4fc2-9c7d-4d1691d4eddd admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:45.642738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 997/2026] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:33:45.648004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba03afb-57dd-4814-a0f9-18bc5ca03ec1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:45.668095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba03afb-57dd-4814-a0f9-18bc5ca03ec1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:45.676100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0470d-502a-40f0-a367-ba6d62cb5bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:45.677169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0470d-502a-40f0-a367-ba6d62cb5bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:45.677658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0470d-502a-40f0-a367-ba6d62cb5bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:45.678129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0470d-502a-40f0-a367-ba6d62cb5bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:45.679341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0470d-502a-40f0-a367-ba6d62cb5bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:45.682476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0470d-502a-40f0-a367-ba6d62cb5bc4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:45.699447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1030/2027] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:45.711957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f319f-0ca3-4fc2-9c7d-4d1691d4eddd admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:45.724180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f319f-0ca3-4fc2-9c7d-4d1691d4eddd admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:45.740497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41610294-1f2f-40a6-8743-dba11cd85ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:45.741632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610294-1f2f-40a6-8743-dba11cd85ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:45.742678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610294-1f2f-40a6-8743-dba11cd85ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:45.743623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610294-1f2f-40a6-8743-dba11cd85ccf admin admin] PATH_INFO: `/v3/projects/17783304fa75491fabf7ed667d6aa24a/users/40ec28ab703c4bd99234969f046d90c3/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:45.780089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41610294-1f2f-40a6-8743-dba11cd85ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=17783304fa75491fabf7ed667d6aa24a, user_id=40ec28ab703c4bd99234969f046d90c3, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:45.785589 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41610294-1f2f-40a6-8743-dba11cd85ccf admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:45.793953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0470d-502a-40f0-a367-ba6d62cb5bc4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:45.796711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-13f0470d-502a-40f0-a367-ba6d62cb5bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:45.813051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0470d-502a-40f0-a367-ba6d62cb5bc4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:45.820337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:45.821155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:45.821502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:45.821785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] PATH_INFO: `/v3/users/6b2936c10531445994423d5fcbe69220` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:45.823000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b2936c10531445994423d5fcbe69220)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:45.825856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:45.913037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 998/2028] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:45 2022] PUT /identity/v3/projects/17783304fa75491fabf7ed667d6aa24a/users/40ec28ab703c4bd99234969f046d90c3/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:45.924201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c9221-4e5f-40ae-b913-35c2382516a7 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:45.924533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c9221-4e5f-40ae-b913-35c2382516a7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:45.924957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c9221-4e5f-40ae-b913-35c2382516a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:45.959582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cf7c9221-4e5f-40ae-b913-35c2382516a7 None None] MFA Rules not processed for user `40ec28ab703c4bd99234969f046d90c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:45.962517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cf7c9221-4e5f-40ae-b913-35c2382516a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:46.047467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 999/2029] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:46.051974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c9221-4e5f-40ae-b913-35c2382516a7 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:46.105560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:46.152318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c9221-4e5f-40ae-b913-35c2382516a7 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:46.162338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031bc3-95fc-4ad2-95a5-63571c8ca47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:46.163620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031bc3-95fc-4ad2-95a5-63571c8ca47f admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:46.164271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031bc3-95fc-4ad2-95a5-63571c8ca47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:46.165059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac031bc3-95fc-4ad2-95a5-63571c8ca47f admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:46.166279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031bc3-95fc-4ad2-95a5-63571c8ca47f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:46.170843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac031bc3-95fc-4ad2-95a5-63571c8ca47f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:46.179504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b2936c10531445994423d5fcbe69220'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.266370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b2936c10531445994423d5fcbe69220'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.281458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b2936c10531445994423d5fcbe69220'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.282233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b2936c10531445994423d5fcbe69220'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.290418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-2a144008-248a-4ada-8bfd-84453e181b55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b2936c10531445994423d5fcbe69220'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:46.298594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1031/2030] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:45 2022] DELETE /identity/v3/users/6b2936c10531445994423d5fcbe69220 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:46.303101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1f828-3349-4787-b392-bcbaac9fd5be None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:46.303386 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1f828-3349-4787-b392-bcbaac9fd5be None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:46.303654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1f828-3349-4787-b392-bcbaac9fd5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:46.324280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f5f1f828-3349-4787-b392-bcbaac9fd5be None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:46.325606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f5f1f828-3349-4787-b392-bcbaac9fd5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:46.365899 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1000/2031] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:46 2022] POST /identity/v3/projects => generated 377 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:33:46.372986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031bc3-95fc-4ad2-95a5-63571c8ca47f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:46.375443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ac031bc3-95fc-4ad2-95a5-63571c8ca47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:46.402600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1032/2032] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:46.418239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1f828-3349-4787-b392-bcbaac9fd5be None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:46.418239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f5f1f828-3349-4787-b392-bcbaac9fd5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:46.455860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:46.462942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac031bc3-95fc-4ad2-95a5-63571c8ca47f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:46.470410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa79a8-6d64-43fe-a7f8-dbdae76e8c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:46.470957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa79a8-6d64-43fe-a7f8-dbdae76e8c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:46.471236 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa79a8-6d64-43fe-a7f8-dbdae76e8c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:46.471498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa79a8-6d64-43fe-a7f8-dbdae76e8c11 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:46.472132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa79a8-6d64-43fe-a7f8-dbdae76e8c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:46.474132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa79a8-6d64-43fe-a7f8-dbdae76e8c11 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:46.490465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1f828-3349-4787-b392-bcbaac9fd5be None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:46.549623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb2266-167b-4eb9-91a8-1013e63e2a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:46.550588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb2266-167b-4eb9-91a8-1013e63e2a09 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:46.551030 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb2266-167b-4eb9-91a8-1013e63e2a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:46.551494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb2266-167b-4eb9-91a8-1013e63e2a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:46.582615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb2266-167b-4eb9-91a8-1013e63e2a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:46.584853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb2266-167b-4eb9-91a8-1013e63e2a09 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:46.600065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1033/2033] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:33:46.634266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1001/2034] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:46 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:33:46.639828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb2266-167b-4eb9-91a8-1013e63e2a09 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:46.655962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb2266-167b-4eb9-91a8-1013e63e2a09 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:46.664304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d6a8b-a693-45ab-a0c8-f32ef6c8292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:46.665289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6a8b-a693-45ab-a0c8-f32ef6c8292b admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:46.665716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6a8b-a693-45ab-a0c8-f32ef6c8292b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:46.666148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6a8b-a693-45ab-a0c8-f32ef6c8292b admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:46.667250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d6a8b-a693-45ab-a0c8-f32ef6c8292b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:46.670058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d6a8b-a693-45ab-a0c8-f32ef6c8292b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:46.687615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1034/2035] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:46.692586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa79a8-6d64-43fe-a7f8-dbdae76e8c11 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:46.709841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa79a8-6d64-43fe-a7f8-dbdae76e8c11 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:46.715324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bafefcb-f7e4-447f-b9f9-cdf118d6900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:46.716575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafefcb-f7e4-447f-b9f9-cdf118d6900e admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:46.716575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafefcb-f7e4-447f-b9f9-cdf118d6900e admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:46.716575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bafefcb-f7e4-447f-b9f9-cdf118d6900e admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:46.717235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafefcb-f7e4-447f-b9f9-cdf118d6900e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:46.718765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bafefcb-f7e4-447f-b9f9-cdf118d6900e admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:46.731546 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1002/2036] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:46.737686 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d6a8b-a693-45ab-a0c8-f32ef6c8292b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:46.758077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d6a8b-a693-45ab-a0c8-f32ef6c8292b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:46.767194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cac872-3b3a-44d8-bdd2-6ce27974d571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:46.768498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cac872-3b3a-44d8-bdd2-6ce27974d571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:46.773185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cac872-3b3a-44d8-bdd2-6ce27974d571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:46.773185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cac872-3b3a-44d8-bdd2-6ce27974d571 admin admin] PATH_INFO: `/v3/projects/a7ac09397bbe43f6b80fc6e58228d9a9/users/cbbd95998cd14b0f808284d8a902ec8d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:46.795721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cac872-3b3a-44d8-bdd2-6ce27974d571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7ac09397bbe43f6b80fc6e58228d9a9, user_id=cbbd95998cd14b0f808284d8a902ec8d, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:46.800269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cac872-3b3a-44d8-bdd2-6ce27974d571 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:46.972943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1035/2037] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:46 2022] PUT /identity/v3/projects/a7ac09397bbe43f6b80fc6e58228d9a9/users/cbbd95998cd14b0f808284d8a902ec8d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:46.996857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34788ce-ac96-4f56-a26a-6fa73afb479e None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:46.996857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34788ce-ac96-4f56-a26a-6fa73afb479e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:46.996857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34788ce-ac96-4f56-a26a-6fa73afb479e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.046726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c34788ce-ac96-4f56-a26a-6fa73afb479e None None] MFA Rules not processed for user `cbbd95998cd14b0f808284d8a902ec8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:47.048904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c34788ce-ac96-4f56-a26a-6fa73afb479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:47.140829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1003/2038] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:47.146372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cac872-3b3a-44d8-bdd2-6ce27974d571 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:47.255163 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cac872-3b3a-44d8-bdd2-6ce27974d571 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:47.261039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f41a9-4d76-474b-914d-815944286bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:47.261584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f41a9-4d76-474b-914d-815944286bac admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.261947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f41a9-4d76-474b-914d-815944286bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.262851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f41a9-4d76-474b-914d-815944286bac admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.263706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f41a9-4d76-474b-914d-815944286bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:47.265798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f41a9-4d76-474b-914d-815944286bac admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:47.474316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34788ce-ac96-4f56-a26a-6fa73afb479e None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:47.476307 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c34788ce-ac96-4f56-a26a-6fa73afb479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:47.510889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34788ce-ac96-4f56-a26a-6fa73afb479e None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:47.510889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:47.511319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.511795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.512296 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] PATH_INFO: `/v3/projects/786b0b5cd1274e269b7dbeecc85c043d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.515050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=786b0b5cd1274e269b7dbeecc85c043d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:47.519014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:47.553316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:47.630489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1036/2039] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:47 2022] POST /identity/v3/projects => generated 377 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:47.637100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f41a9-4d76-474b-914d-815944286bac admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:47.716597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f41a9-4d76-474b-914d-815944286bac admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:47.721414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be487945-26a5-4105-b2d6-c4e364b3137e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:47.721933 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be487945-26a5-4105-b2d6-c4e364b3137e admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:47.722209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be487945-26a5-4105-b2d6-c4e364b3137e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:47.722478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be487945-26a5-4105-b2d6-c4e364b3137e admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:47.723112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be487945-26a5-4105-b2d6-c4e364b3137e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:47.725062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be487945-26a5-4105-b2d6-c4e364b3137e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:47.872328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '786b0b5cd1274e269b7dbeecc85c043d', 'request_id': 'req-a5065070-952b-4897-ad73-c891f566d687'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:47.878020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '786b0b5cd1274e269b7dbeecc85c043d', 'request_id': 'req-a5065070-952b-4897-ad73-c891f566d687'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:47.922352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1037/2040] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:47 2022] POST /identity/v3/users => generated 295 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:33:47.932540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be487945-26a5-4105-b2d6-c4e364b3137e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:47.948506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be487945-26a5-4105-b2d6-c4e364b3137e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.030596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ecd8d-1efb-4ff3-be66-0e35b618790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.031202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ecd8d-1efb-4ff3-be66-0e35b618790e admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.031479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ecd8d-1efb-4ff3-be66-0e35b618790e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.031752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ecd8d-1efb-4ff3-be66-0e35b618790e admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.032399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ecd8d-1efb-4ff3-be66-0e35b618790e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.034050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ecd8d-1efb-4ff3-be66-0e35b618790e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.048029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1038/2041] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:48.048977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '786b0b5cd1274e269b7dbeecc85c043d', 'request_id': 'req-a5065070-952b-4897-ad73-c891f566d687'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:48.056507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ecd8d-1efb-4ff3-be66-0e35b618790e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.066547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] The token cache is being invalidate because project 786b0b5cd1274e269b7dbeecc85c043d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:33:48.067058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:48.068424 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:48.069963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ecd8d-1efb-4ff3-be66-0e35b618790e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.071084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1004/2042] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:33:47 2022] DELETE /identity/v3/projects/786b0b5cd1274e269b7dbeecc85c043d => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:33:48.076746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.077334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8694c7-2f1e-4659-8226-bf2eee701dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.078152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8694c7-2f1e-4659-8226-bf2eee701dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.078577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8694c7-2f1e-4659-8226-bf2eee701dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.078772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:48.079034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8694c7-2f1e-4659-8226-bf2eee701dc5 admin admin] PATH_INFO: `/v3/projects/ad3e75b3082f462bb6b1435d8ee3623c/users/d6b81e1ce1c84c9285192ca7fe38bef9/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.101781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5065070-952b-4897-ad73-c891f566d687 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.105783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8694c7-2f1e-4659-8226-bf2eee701dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad3e75b3082f462bb6b1435d8ee3623c, user_id=d6b81e1ce1c84c9285192ca7fe38bef9, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.109223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.114553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.114553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.114553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] PATH_INFO: `/v3/users/c66f476eebd84c1dad21a6866c625aed` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.114976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8694c7-2f1e-4659-8226-bf2eee701dc5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.115125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66f476eebd84c1dad21a6866c625aed)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.115125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.164787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:48.279781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1039/2043] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:48 2022] PUT /identity/v3/projects/ad3e75b3082f462bb6b1435d8ee3623c/users/d6b81e1ce1c84c9285192ca7fe38bef9/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:33:48.284411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8694c7-2f1e-4659-8226-bf2eee701dc5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.286175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fe8694c7-2f1e-4659-8226-bf2eee701dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:48.386179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8694c7-2f1e-4659-8226-bf2eee701dc5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.393257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772a6d1-bbf6-4d36-b1cf-ec9e68ec115e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.393882 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772a6d1-bbf6-4d36-b1cf-ec9e68ec115e admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.394170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772a6d1-bbf6-4d36-b1cf-ec9e68ec115e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.394436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772a6d1-bbf6-4d36-b1cf-ec9e68ec115e admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.395060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772a6d1-bbf6-4d36-b1cf-ec9e68ec115e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.396873 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772a6d1-bbf6-4d36-b1cf-ec9e68ec115e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.408148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1040/2044] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:33:48.412670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772a6d1-bbf6-4d36-b1cf-ec9e68ec115e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.426008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772a6d1-bbf6-4d36-b1cf-ec9e68ec115e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.436832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2ea1e-1668-4f4d-8ca8-fcd6227ed64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.441224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2ea1e-1668-4f4d-8ca8-fcd6227ed64d admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.441687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2ea1e-1668-4f4d-8ca8-fcd6227ed64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.442165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2ea1e-1668-4f4d-8ca8-fcd6227ed64d admin admin] PATH_INFO: `/v3/projects/ad3e75b3082f462bb6b1435d8ee3623c/users/d6b81e1ce1c84c9285192ca7fe38bef9/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.445650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2ea1e-1668-4f4d-8ca8-fcd6227ed64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad3e75b3082f462bb6b1435d8ee3623c, user_id=d6b81e1ce1c84c9285192ca7fe38bef9, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.450204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2ea1e-1668-4f4d-8ca8-fcd6227ed64d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.468498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1041/2045] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:48 2022] PUT /identity/v3/projects/ad3e75b3082f462bb6b1435d8ee3623c/users/d6b81e1ce1c84c9285192ca7fe38bef9/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:33:48.480658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cc4c2-2e10-48b0-9b16-2d2f52a2b4ee None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.482563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cc4c2-2e10-48b0-9b16-2d2f52a2b4ee None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.482889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7cc4c2-2e10-48b0-9b16-2d2f52a2b4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.498879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66f476eebd84c1dad21a6866c625aed'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:48.509897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66f476eebd84c1dad21a6866c625aed'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:48.509897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66f476eebd84c1dad21a6866c625aed'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:48.511005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66f476eebd84c1dad21a6866c625aed'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:48.524622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6a7cc4c2-2e10-48b0-9b16-2d2f52a2b4ee None None] MFA Rules not processed for user `d6b81e1ce1c84c9285192ca7fe38bef9`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:48.526682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6a7cc4c2-2e10-48b0-9b16-2d2f52a2b4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:48.533376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66f476eebd84c1dad21a6866c625aed'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:48.625117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1042/2046] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:48.637795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cc4c2-2e10-48b0-9b16-2d2f52a2b4ee None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:48.639194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6a7cc4c2-2e10-48b0-9b16-2d2f52a2b4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:48.716080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7cc4c2-2e10-48b0-9b16-2d2f52a2b4ee None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:48.720299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725b17f-a86a-42f8-b900-be36d12f6659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:48.720948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725b17f-a86a-42f8-b900-be36d12f6659 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:48.721223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725b17f-a86a-42f8-b900-be36d12f6659 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:48.721489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725b17f-a86a-42f8-b900-be36d12f6659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:48.723268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b725b17f-a86a-42f8-b900-be36d12f6659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:48.751020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1005/2047] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:33:48 2022] DELETE /identity/v3/users/c66f476eebd84c1dad21a6866c625aed => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:33:48.807286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725b17f-a86a-42f8-b900-be36d12f6659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:48.809306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725b17f-a86a-42f8-b900-be36d12f6659 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:48.820037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1043/2048] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:33:49.958048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:49.959846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:50.040548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f6420-7047-4633-909f-b566ce61870c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:50.045199 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:50.045826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:50.046101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:50.046450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] PATH_INFO: `/v3/projects/835432b860ec471db3019169348ee742` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:50.047603 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=835432b860ec471db3019169348ee742)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:50.049539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:50.220301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835432b860ec471db3019169348ee742', 'request_id': 'req-6dd6b6a0-32e4-405f-b141-e0d374b14165'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:50.226343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835432b860ec471db3019169348ee742', 'request_id': 'req-6dd6b6a0-32e4-405f-b141-e0d374b14165'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:50.460331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835432b860ec471db3019169348ee742', 'request_id': 'req-6dd6b6a0-32e4-405f-b141-e0d374b14165'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:50.475370 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] The token cache is being invalidate because project 835432b860ec471db3019169348ee742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:33:50.475827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:50.476846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6dd6b6a0-32e4-405f-b141-e0d374b14165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:33:50.479240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1006/2049] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:33:49 2022] DELETE /identity/v3/projects/835432b860ec471db3019169348ee742 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:33:50.484468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcf426-bea6-4307-aa84-aa2abe8e0c6e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:50.485065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcf426-bea6-4307-aa84-aa2abe8e0c6e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:50.485065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcf426-bea6-4307-aa84-aa2abe8e0c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:50.506680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f9bcf426-bea6-4307-aa84-aa2abe8e0c6e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:50.507880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f9bcf426-bea6-4307-aa84-aa2abe8e0c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:50.580413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1044/2050] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:50.589813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99655-3a5e-44d5-99c3-5ae157ca6b56 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:50.590123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99655-3a5e-44d5-99c3-5ae157ca6b56 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:50.590394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da99655-3a5e-44d5-99c3-5ae157ca6b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:50.615967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-0da99655-3a5e-44d5-99c3-5ae157ca6b56 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:50.617481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0da99655-3a5e-44d5-99c3-5ae157ca6b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:50.627822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1007/2051] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:33:50.633450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcf426-bea6-4307-aa84-aa2abe8e0c6e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:50.644248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcf426-bea6-4307-aa84-aa2abe8e0c6e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:50.649073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba639000-ee5c-4d16-bab5-c807da43a99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:50.649673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba639000-ee5c-4d16-bab5-c807da43a99a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:50.649947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba639000-ee5c-4d16-bab5-c807da43a99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:50.650217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba639000-ee5c-4d16-bab5-c807da43a99a admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:50.650862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba639000-ee5c-4d16-bab5-c807da43a99a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:33:50.651160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba639000-ee5c-4d16-bab5-c807da43a99a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:50.652971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba639000-ee5c-4d16-bab5-c807da43a99a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:50.665914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:50.666277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1045/2052] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:33:50.671930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99655-3a5e-44d5-99c3-5ae157ca6b56 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:50.681727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da99655-3a5e-44d5-99c3-5ae157ca6b56 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:50.688043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76678d93-5b7a-4410-aa4f-79b6ac5cb06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:50.703421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76678d93-5b7a-4410-aa4f-79b6ac5cb06c admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:50.703421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76678d93-5b7a-4410-aa4f-79b6ac5cb06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:50.703421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76678d93-5b7a-4410-aa4f-79b6ac5cb06c admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:50.703421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76678d93-5b7a-4410-aa4f-79b6ac5cb06c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:50.703421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76678d93-5b7a-4410-aa4f-79b6ac5cb06c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:50.858865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1008/2053] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:33:50 2022] POST /identity/v3/projects => generated 399 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:50.860126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba639000-ee5c-4d16-bab5-c807da43a99a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:50.870730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba639000-ee5c-4d16-bab5-c807da43a99a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:50.883994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:50.942393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fba3d4-ec23-492f-9c71-db26e9b4ae41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:50.943152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba3d4-ec23-492f-9c71-db26e9b4ae41 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:50.943524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba3d4-ec23-492f-9c71-db26e9b4ae41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:50.944141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba3d4-ec23-492f-9c71-db26e9b4ae41 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:50.945350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fba3d4-ec23-492f-9c71-db26e9b4ae41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:50.947402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fba3d4-ec23-492f-9c71-db26e9b4ae41 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:51.060717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1046/2054] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:33:50 2022] POST /identity/v3/users => generated 300 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:33:51.074466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76678d93-5b7a-4410-aa4f-79b6ac5cb06c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:51.076318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76678d93-5b7a-4410-aa4f-79b6ac5cb06c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:51.081362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ba8fb-43a3-41c5-bb48-689b55ff50a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:51.081982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ba8fb-43a3-41c5-bb48-689b55ff50a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:51.082263 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ba8fb-43a3-41c5-bb48-689b55ff50a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:51.082538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ba8fb-43a3-41c5-bb48-689b55ff50a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:51.083185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ba8fb-43a3-41c5-bb48-689b55ff50a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:51.084828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ba8fb-43a3-41c5-bb48-689b55ff50a5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:51.095511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1009/2055] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:33:51.099621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fba3d4-ec23-492f-9c71-db26e9b4ae41 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:51.110524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fba3d4-ec23-492f-9c71-db26e9b4ae41 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:51.115191 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a0c34-b5bf-4d87-8766-01a5f1844141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:51.115795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a0c34-b5bf-4d87-8766-01a5f1844141 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:51.116073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a0c34-b5bf-4d87-8766-01a5f1844141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:51.116347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a0c34-b5bf-4d87-8766-01a5f1844141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:51.117043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a0c34-b5bf-4d87-8766-01a5f1844141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:51.118672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a0c34-b5bf-4d87-8766-01a5f1844141 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:51.131088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1047/2056] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:33:51.135384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ba8fb-43a3-41c5-bb48-689b55ff50a5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:51.146145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ba8fb-43a3-41c5-bb48-689b55ff50a5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:51.158342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010290ab-eec6-4d04-82cb-b3f7f0658a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:51.158342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010290ab-eec6-4d04-82cb-b3f7f0658a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:51.158342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010290ab-eec6-4d04-82cb-b3f7f0658a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:51.158342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010290ab-eec6-4d04-82cb-b3f7f0658a76 admin admin] PATH_INFO: `/v3/projects/9861a0f44d6847b185d49c65549363a7/users/d5d9aaa0741f4b6da4cdce767ed7f79c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:51.167373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010290ab-eec6-4d04-82cb-b3f7f0658a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9861a0f44d6847b185d49c65549363a7, user_id=d5d9aaa0741f4b6da4cdce767ed7f79c, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:51.170371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010290ab-eec6-4d04-82cb-b3f7f0658a76 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:51.255128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1010/2057] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:33:51 2022] PUT /identity/v3/projects/9861a0f44d6847b185d49c65549363a7/users/d5d9aaa0741f4b6da4cdce767ed7f79c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:33:51.266079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0788-c382-4f4b-90b3-c083b548b6d0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:51.266357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0788-c382-4f4b-90b3-c083b548b6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:51.266626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa0788-c382-4f4b-90b3-c083b548b6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:51.297279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-83fa0788-c382-4f4b-90b3-c083b548b6d0 None None] MFA Rules not processed for user `d5d9aaa0741f4b6da4cdce767ed7f79c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:33:51.298426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-83fa0788-c382-4f4b-90b3-c083b548b6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:51.365409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1048/2058] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:33:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:33:51.382264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010290ab-eec6-4d04-82cb-b3f7f0658a76 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:51.384123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-010290ab-eec6-4d04-82cb-b3f7f0658a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:51.443421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010290ab-eec6-4d04-82cb-b3f7f0658a76 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:51.447145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e507e-f237-45b5-b086-acd8544af2eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:51.447752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e507e-f237-45b5-b086-acd8544af2eb service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:51.448031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e507e-f237-45b5-b086-acd8544af2eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:51.448305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e507e-f237-45b5-b086-acd8544af2eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:51.457386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e507e-f237-45b5-b086-acd8544af2eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:51.458833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e507e-f237-45b5-b086-acd8544af2eb service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:51.468291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1011/2059] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:33:51.655419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:58.290817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa0788-c382-4f4b-90b3-c083b548b6d0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:33:58.302785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa0788-c382-4f4b-90b3-c083b548b6d0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:33:58.316118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473a6ec-c06a-4874-992c-4f7e615d3181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:33:58.316118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473a6ec-c06a-4874-992c-4f7e615d3181 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:33:58.316118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473a6ec-c06a-4874-992c-4f7e615d3181 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:33:58.316118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473a6ec-c06a-4874-992c-4f7e615d3181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:33:58.316453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f473a6ec-c06a-4874-992c-4f7e615d3181 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:33:58.385317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473a6ec-c06a-4874-992c-4f7e615d3181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:33:58.387945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473a6ec-c06a-4874-992c-4f7e615d3181 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:33:58.392116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:33:58.402329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1049/2060] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:33:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:01.775130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:02.087691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e507e-f237-45b5-b086-acd8544af2eb service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:02.099427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e507e-f237-45b5-b086-acd8544af2eb service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:02.106313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:02.107277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:02.107747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:02.108087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:02.109904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:02.187360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:02.188811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:02.198768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1012/2061] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:09.071347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:10.066295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473a6ec-c06a-4874-992c-4f7e615d3181 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:10.087144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:10.088642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:10.159950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473a6ec-c06a-4874-992c-4f7e615d3181 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:10.165235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:10.166708 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:10.167040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:10.167894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] PATH_INFO: `/v3/users/d5d9aaa0741f4b6da4cdce767ed7f79c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:10.185608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d9aaa0741f4b6da4cdce767ed7f79c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:10.185608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:10.206671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d515db-2285-46e5-a5c6-fbfcca19028d service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:10.216169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:10.217426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:10.218037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:10.218595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] PATH_INFO: `/v3/users/242a0be3cc7c49a39703f6ceabc36542` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:10.220397 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] RBAC: Authorizing `identity:delete_user(user_id=242a0be3cc7c49a39703f6ceabc36542)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:10.223471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:10.438071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d9aaa0741f4b6da4cdce767ed7f79c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.473936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242a0be3cc7c49a39703f6ceabc36542'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.474604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242a0be3cc7c49a39703f6ceabc36542'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.476426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242a0be3cc7c49a39703f6ceabc36542'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.487491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242a0be3cc7c49a39703f6ceabc36542'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.508913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-c286c022-40f5-4e48-a0b2-16cc5ff6dbac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242a0be3cc7c49a39703f6ceabc36542'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.597437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d9aaa0741f4b6da4cdce767ed7f79c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.628610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d9aaa0741f4b6da4cdce767ed7f79c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.629689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d9aaa0741f4b6da4cdce767ed7f79c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.637656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-8e61b329-46cf-42d4-a7b2-1cf2baeb5d66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d9aaa0741f4b6da4cdce767ed7f79c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:10.643892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1050/2062] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:34:10 2022] DELETE /identity/v3/users/d5d9aaa0741f4b6da4cdce767ed7f79c => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:10.654651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb3065-64a4-4a1c-a2d8-0de0d8d743d8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:10.654936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb3065-64a4-4a1c-a2d8-0de0d8d743d8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:10.655203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb3065-64a4-4a1c-a2d8-0de0d8d743d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:10.687618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c8bb3065-64a4-4a1c-a2d8-0de0d8d743d8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:10.688894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c8bb3065-64a4-4a1c-a2d8-0de0d8d743d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:10.759289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1051/2063] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:34:10.770117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb3065-64a4-4a1c-a2d8-0de0d8d743d8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:10.771988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c8bb3065-64a4-4a1c-a2d8-0de0d8d743d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:10.785789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1013/2064] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:34:10 2022] DELETE /identity/v3/users/242a0be3cc7c49a39703f6ceabc36542 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:10.800212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6d7d7-caee-43a8-b3d7-0e6d84c19ace None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:10.800723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6d7d7-caee-43a8-b3d7-0e6d84c19ace None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:10.801950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6d7d7-caee-43a8-b3d7-0e6d84c19ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:10.802888 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:10.834944 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-04b6d7d7-caee-43a8-b3d7-0e6d84c19ace None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:10.836329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-04b6d7d7-caee-43a8-b3d7-0e6d84c19ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:10.848921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:10.857560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb3065-64a4-4a1c-a2d8-0de0d8d743d8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:10.928419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cb77e-8c0a-4c0e-8d32-503a76f6ca73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:10.930395 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cb77e-8c0a-4c0e-8d32-503a76f6ca73 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:10.930673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cb77e-8c0a-4c0e-8d32-503a76f6ca73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:10.930952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cb77e-8c0a-4c0e-8d32-503a76f6ca73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:10.956102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1014/2065] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:34:10.968281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6d7d7-caee-43a8-b3d7-0e6d84c19ace None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:10.976311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-04b6d7d7-caee-43a8-b3d7-0e6d84c19ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:10.983920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6d7d7-caee-43a8-b3d7-0e6d84c19ace None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:10.987718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9956592-0b80-4b6f-8034-392ef0ce834b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:10.988347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9956592-0b80-4b6f-8034-392ef0ce834b service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:10.988627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9956592-0b80-4b6f-8034-392ef0ce834b service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:10.988937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9956592-0b80-4b6f-8034-392ef0ce834b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:10.999863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9956592-0b80-4b6f-8034-392ef0ce834b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:11.001348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9956592-0b80-4b6f-8034-392ef0ce834b service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:11.019603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cb77e-8c0a-4c0e-8d32-503a76f6ca73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:11.020861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1015/2066] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:34:11.022014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cb77e-8c0a-4c0e-8d32-503a76f6ca73 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:11.043709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1052/2067] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:34:11.763021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9956592-0b80-4b6f-8034-392ef0ce834b service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:11.763021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c9956592-0b80-4b6f-8034-392ef0ce834b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:11.764404 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cb77e-8c0a-4c0e-8d32-503a76f6ca73 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:11.767211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0d8cb77e-8c0a-4c0e-8d32-503a76f6ca73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:11.785819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9956592-0b80-4b6f-8034-392ef0ce834b service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:11.786362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cb77e-8c0a-4c0e-8d32-503a76f6ca73 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:11.793459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:11.794217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:11.794600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:11.794983 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] PATH_INFO: `/v3/projects/9861a0f44d6847b185d49c65549363a7` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:11.796353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9861a0f44d6847b185d49c65549363a7)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:11.799950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:11.800190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:11.800299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:11.800299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:11.800299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] PATH_INFO: `/v3/projects/d92b23b5037846c18c874be807df3e39` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:11.801495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92b23b5037846c18c874be807df3e39)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:11.803599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:12.039349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9861a0f44d6847b185d49c65549363a7', 'request_id': 'req-7c0aa140-5630-4527-86bf-d34f19ac8205'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.040902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92b23b5037846c18c874be807df3e39', 'request_id': 'req-90eb43fe-7db8-4efd-99e2-b39da757f191'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.047874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9861a0f44d6847b185d49c65549363a7', 'request_id': 'req-7c0aa140-5630-4527-86bf-d34f19ac8205'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.054688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92b23b5037846c18c874be807df3e39', 'request_id': 'req-90eb43fe-7db8-4efd-99e2-b39da757f191'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.191803 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9861a0f44d6847b185d49c65549363a7', 'request_id': 'req-7c0aa140-5630-4527-86bf-d34f19ac8205'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.207176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] The token cache is being invalidate because project 9861a0f44d6847b185d49c65549363a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:34:12.207499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.208247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.210013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1053/2068] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:34:11 2022] DELETE /identity/v3/projects/9861a0f44d6847b185d49c65549363a7 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:12.328051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92b23b5037846c18c874be807df3e39', 'request_id': 'req-90eb43fe-7db8-4efd-99e2-b39da757f191'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.346567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] The token cache is being invalidate because project d92b23b5037846c18c874be807df3e39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:34:12.346889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.347670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.349783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1016/2069] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:34:11 2022] DELETE /identity/v3/projects/d92b23b5037846c18c874be807df3e39 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:12.353782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:12.356740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:12.371278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:12.454725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0aa140-5630-4527-86bf-d34f19ac8205 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:12.468398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:12.469183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:12.469466 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:12.469785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] PATH_INFO: `/v3/users/e0b9222140dd4dd1835f318ed5a402aa` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:12.471017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b9222140dd4dd1835f318ed5a402aa)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:12.473018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:12.843598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b9222140dd4dd1835f318ed5a402aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.981115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b9222140dd4dd1835f318ed5a402aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.994629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b9222140dd4dd1835f318ed5a402aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:12.996745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b9222140dd4dd1835f318ed5a402aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:13.008636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b9222140dd4dd1835f318ed5a402aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:13.016272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1054/2070] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:34:12 2022] DELETE /identity/v3/users/e0b9222140dd4dd1835f318ed5a402aa => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:13.662015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:13.664536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:13.747339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb43fe-7db8-4efd-99e2-b39da757f191 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:13.777859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:13.805204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:13.805839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:13.806114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:13.806424 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] PATH_INFO: `/v3/projects/dd2e3442b7cf4d7cbbd05c7838e880eb` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:13.807842 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd2e3442b7cf4d7cbbd05c7838e880eb)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:13.810144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:13.985966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:13.987414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:14.001646 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d299e-9110-42ae-aeee-3700cbb1a589 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:14.006307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:14.006981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.007257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.007525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] PATH_INFO: `/v3/users/40ec28ab703c4bd99234969f046d90c3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.008644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ec28ab703c4bd99234969f046d90c3)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:14.010589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:14.089666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd2e3442b7cf4d7cbbd05c7838e880eb', 'request_id': 'req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.095273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd2e3442b7cf4d7cbbd05c7838e880eb', 'request_id': 'req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.277002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd2e3442b7cf4d7cbbd05c7838e880eb', 'request_id': 'req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.290639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] The token cache is being invalidate because project dd2e3442b7cf4d7cbbd05c7838e880eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:34:14.290926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.291661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-995de1fc-7dc7-4cc9-a701-a39e64e0fba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.293316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1017/2071] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:34:13 2022] DELETE /identity/v3/projects/dd2e3442b7cf4d7cbbd05c7838e880eb => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:14.298172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbeeb3-0335-43ac-9ad9-3dae9bf30607 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.298458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbeeb3-0335-43ac-9ad9-3dae9bf30607 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.298734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbeeb3-0335-43ac-9ad9-3dae9bf30607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.321271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-dffbeeb3-0335-43ac-9ad9-3dae9bf30607 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:14.322632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-dffbeeb3-0335-43ac-9ad9-3dae9bf30607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:14.372952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ec28ab703c4bd99234969f046d90c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.385251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1018/2072] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:34:14.393724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d066a8b-9dcf-4ed1-955c-6c1b9ee00b84 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.394010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d066a8b-9dcf-4ed1-955c-6c1b9ee00b84 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.394285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d066a8b-9dcf-4ed1-955c-6c1b9ee00b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.414322 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9d066a8b-9dcf-4ed1-955c-6c1b9ee00b84 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:14.415514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9d066a8b-9dcf-4ed1-955c-6c1b9ee00b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:14.423974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1019/2073] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:34:14.428564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d066a8b-9dcf-4ed1-955c-6c1b9ee00b84 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:14.439862 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d066a8b-9dcf-4ed1-955c-6c1b9ee00b84 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:14.444716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9cacf-272d-419b-8d3d-0e73762ba27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:14.449043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9cacf-272d-419b-8d3d-0e73762ba27b admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.449043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9cacf-272d-419b-8d3d-0e73762ba27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.449043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9cacf-272d-419b-8d3d-0e73762ba27b admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.449043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9cacf-272d-419b-8d3d-0e73762ba27b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:34:14.449043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9cacf-272d-419b-8d3d-0e73762ba27b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:14.449875 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9cacf-272d-419b-8d3d-0e73762ba27b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:14.466482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1020/2074] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:34:14.467598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9cacf-272d-419b-8d3d-0e73762ba27b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:14.477722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9cacf-272d-419b-8d3d-0e73762ba27b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:14.483249 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0dea7-6ada-4ee4-a95b-c2f05d62a4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:14.483857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0dea7-6ada-4ee4-a95b-c2f05d62a4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.484131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0dea7-6ada-4ee4-a95b-c2f05d62a4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.484396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0dea7-6ada-4ee4-a95b-c2f05d62a4c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.485075 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0dea7-6ada-4ee4-a95b-c2f05d62a4c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:14.489191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0dea7-6ada-4ee4-a95b-c2f05d62a4c5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:14.550842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ec28ab703c4bd99234969f046d90c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.571511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ec28ab703c4bd99234969f046d90c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.572323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ec28ab703c4bd99234969f046d90c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.580120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d16f4e27-d692-4c7e-ba8a-d8c5950069b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ec28ab703c4bd99234969f046d90c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:14.582474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:14.588922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1055/2075] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:34:13 2022] DELETE /identity/v3/users/40ec28ab703c4bd99234969f046d90c3 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:14.597160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099c223-10c9-4282-a47d-003b02af1d23 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.597447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099c223-10c9-4282-a47d-003b02af1d23 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.597717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099c223-10c9-4282-a47d-003b02af1d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.617708 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5099c223-10c9-4282-a47d-003b02af1d23 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:14.619111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5099c223-10c9-4282-a47d-003b02af1d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:14.640354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1021/2076] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:34:14 2022] POST /identity/v3/projects => generated 369 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:34:14.650939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0dea7-6ada-4ee4-a95b-c2f05d62a4c5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:14.657104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-66c0dea7-6ada-4ee4-a95b-c2f05d62a4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:14.680445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:14.689859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1056/2077] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:34:14.700487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099c223-10c9-4282-a47d-003b02af1d23 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:14.701864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5099c223-10c9-4282-a47d-003b02af1d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:14.727363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0dea7-6ada-4ee4-a95b-c2f05d62a4c5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:14.737807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04d6db-6e81-4538-a114-937fc121ab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:14.738450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04d6db-6e81-4538-a114-937fc121ab00 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.738726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04d6db-6e81-4538-a114-937fc121ab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.738999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04d6db-6e81-4538-a114-937fc121ab00 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.739661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04d6db-6e81-4538-a114-937fc121ab00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:14.741640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04d6db-6e81-4538-a114-937fc121ab00 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:14.770351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099c223-10c9-4282-a47d-003b02af1d23 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:14.822684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b00c9-3fcc-4fcb-afe7-f06608478d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:14.823955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b00c9-3fcc-4fcb-afe7-f06608478d28 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.824243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b00c9-3fcc-4fcb-afe7-f06608478d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.824512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b00c9-3fcc-4fcb-afe7-f06608478d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.834642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b00c9-3fcc-4fcb-afe7-f06608478d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:14.836212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b00c9-3fcc-4fcb-afe7-f06608478d28 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:14.846460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1057/2078] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:34:14.856609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1022/2079] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:34:14 2022] POST /identity/v3/users => generated 285 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:34:14.857917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b00c9-3fcc-4fcb-afe7-f06608478d28 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:14.868568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b00c9-3fcc-4fcb-afe7-f06608478d28 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:14.874576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db61ea-84cb-4a90-99f3-298577204549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:14.875187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db61ea-84cb-4a90-99f3-298577204549 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.875458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db61ea-84cb-4a90-99f3-298577204549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.875726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db61ea-84cb-4a90-99f3-298577204549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.876374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db61ea-84cb-4a90-99f3-298577204549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:14.878054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db61ea-84cb-4a90-99f3-298577204549 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:14.899874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1058/2080] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:34:14.900046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04d6db-6e81-4538-a114-937fc121ab00 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:14.906481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04d6db-6e81-4538-a114-937fc121ab00 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:14.913122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a1f7c-36d6-4f04-a81f-64af2712cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:14.914318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a1f7c-36d6-4f04-a81f-64af2712cc3d admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.915189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a1f7c-36d6-4f04-a81f-64af2712cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.915606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a1f7c-36d6-4f04-a81f-64af2712cc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:14.916947 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a1f7c-36d6-4f04-a81f-64af2712cc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:14.921711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a1f7c-36d6-4f04-a81f-64af2712cc3d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:14.937748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1023/2081] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:34:14.942117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db61ea-84cb-4a90-99f3-298577204549 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:14.964351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db61ea-84cb-4a90-99f3-298577204549 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:14.972479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d218b5-5c2e-4c82-a6bf-ffe37c5cfb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:14.973775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d218b5-5c2e-4c82-a6bf-ffe37c5cfb6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:14.974266 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d218b5-5c2e-4c82-a6bf-ffe37c5cfb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:14.974568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d218b5-5c2e-4c82-a6bf-ffe37c5cfb6e admin admin] PATH_INFO: `/v3/projects/c6f81b64c0e54ba0b1122d884b2d703e/users/133c7468b9e947498ab1e90de7b7b9e0/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:15.006604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d218b5-5c2e-4c82-a6bf-ffe37c5cfb6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f81b64c0e54ba0b1122d884b2d703e, user_id=133c7468b9e947498ab1e90de7b7b9e0, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:15.009308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d218b5-5c2e-4c82-a6bf-ffe37c5cfb6e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:15.164728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1059/2082] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:34:14 2022] PUT /identity/v3/projects/c6f81b64c0e54ba0b1122d884b2d703e/users/133c7468b9e947498ab1e90de7b7b9e0/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:15.185856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedc185-4243-4cb1-9e19-6ab6b451dc39 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:15.186343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedc185-4243-4cb1-9e19-6ab6b451dc39 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:15.186892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedc185-4243-4cb1-9e19-6ab6b451dc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:15.223374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cbedc185-4243-4cb1-9e19-6ab6b451dc39 None None] MFA Rules not processed for user `133c7468b9e947498ab1e90de7b7b9e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:15.224882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cbedc185-4243-4cb1-9e19-6ab6b451dc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:15.303123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1024/2083] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:34:15.315160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d218b5-5c2e-4c82-a6bf-ffe37c5cfb6e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:15.351059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:15.391982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d218b5-5c2e-4c82-a6bf-ffe37c5cfb6e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:15.408369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175408a9-774e-4bd2-bafd-a2ea2a68a517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:15.409008 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175408a9-774e-4bd2-bafd-a2ea2a68a517 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:15.409282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175408a9-774e-4bd2-bafd-a2ea2a68a517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:15.409549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175408a9-774e-4bd2-bafd-a2ea2a68a517 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:15.410186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175408a9-774e-4bd2-bafd-a2ea2a68a517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:15.412134 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175408a9-774e-4bd2-bafd-a2ea2a68a517 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:15.428033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedc185-4243-4cb1-9e19-6ab6b451dc39 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:15.430153 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cbedc185-4243-4cb1-9e19-6ab6b451dc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:15.446916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedc185-4243-4cb1-9e19-6ab6b451dc39 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:15.453389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:15.454066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:15.454345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:15.454615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] PATH_INFO: `/v3/projects/17783304fa75491fabf7ed667d6aa24a` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:15.455743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17783304fa75491fabf7ed667d6aa24a)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:15.457704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:15.622380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1060/2084] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:34:15 2022] POST /identity/v3/projects => generated 367 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:34:15.626446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175408a9-774e-4bd2-bafd-a2ea2a68a517 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:15.690052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17783304fa75491fabf7ed667d6aa24a', 'request_id': 'req-9e16310c-258c-44b7-8c74-eeca63d991d9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:15.696497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17783304fa75491fabf7ed667d6aa24a', 'request_id': 'req-9e16310c-258c-44b7-8c74-eeca63d991d9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:15.698987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175408a9-774e-4bd2-bafd-a2ea2a68a517 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:15.770407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b503e4-fe2a-4cc7-b79a-8c5c4187335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:15.771515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b503e4-fe2a-4cc7-b79a-8c5c4187335d admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:15.772068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b503e4-fe2a-4cc7-b79a-8c5c4187335d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:15.772549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b503e4-fe2a-4cc7-b79a-8c5c4187335d admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:15.773735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b503e4-fe2a-4cc7-b79a-8c5c4187335d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:15.778865 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b503e4-fe2a-4cc7-b79a-8c5c4187335d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:15.837024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17783304fa75491fabf7ed667d6aa24a', 'request_id': 'req-9e16310c-258c-44b7-8c74-eeca63d991d9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:15.851204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] The token cache is being invalidate because project 17783304fa75491fabf7ed667d6aa24a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:34:15.851481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:15.852178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:15.853928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1025/2085] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:34:15 2022] DELETE /identity/v3/projects/17783304fa75491fabf7ed667d6aa24a => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:15.858697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:15.860116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:15.875501 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16310c-258c-44b7-8c74-eeca63d991d9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:15.882032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:15.882698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:15.883080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:15.883358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] PATH_INFO: `/v3/users/cbbd95998cd14b0f808284d8a902ec8d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:15.885179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbbd95998cd14b0f808284d8a902ec8d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:15.887444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:15.893912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:15.940994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1061/2086] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:34:15 2022] POST /identity/v3/users => generated 290 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:34:15.947036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b503e4-fe2a-4cc7-b79a-8c5c4187335d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:15.951220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b1b503e4-fe2a-4cc7-b79a-8c5c4187335d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:15.964788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b503e4-fe2a-4cc7-b79a-8c5c4187335d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:15.973919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d92d7-0dd3-441a-91cb-f893ef419be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:15.974730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d92d7-0dd3-441a-91cb-f893ef419be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:15.975032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d92d7-0dd3-441a-91cb-f893ef419be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:15.975597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d92d7-0dd3-441a-91cb-f893ef419be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:15.976403 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d92d7-0dd3-441a-91cb-f893ef419be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:15.978334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d92d7-0dd3-441a-91cb-f893ef419be1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:15.991333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1062/2087] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:34:15.996608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d92d7-0dd3-441a-91cb-f893ef419be1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:16.007243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d92d7-0dd3-441a-91cb-f893ef419be1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:16.023641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bf8cf-ac3f-474f-b993-3beaf5b2f65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:16.024723 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bf8cf-ac3f-474f-b993-3beaf5b2f65a admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:16.025065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bf8cf-ac3f-474f-b993-3beaf5b2f65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:16.025452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bf8cf-ac3f-474f-b993-3beaf5b2f65a admin admin] PATH_INFO: `/v3/projects/2cc7af711ad84c7bbcbc80989bf59c86/users/f339012f3b154a88b9c9e9a96ebfa88f/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:16.043959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bf8cf-ac3f-474f-b993-3beaf5b2f65a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc7af711ad84c7bbcbc80989bf59c86, user_id=f339012f3b154a88b9c9e9a96ebfa88f, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:16.046653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82bf8cf-ac3f-474f-b993-3beaf5b2f65a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:16.204430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1063/2088] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:34:15 2022] PUT /identity/v3/projects/2cc7af711ad84c7bbcbc80989bf59c86/users/f339012f3b154a88b9c9e9a96ebfa88f/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:16.208688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bf8cf-ac3f-474f-b993-3beaf5b2f65a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:16.217168 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbbd95998cd14b0f808284d8a902ec8d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:16.218569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbbd95998cd14b0f808284d8a902ec8d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:16.219066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbbd95998cd14b0f808284d8a902ec8d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:16.228116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:16.231391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbbd95998cd14b0f808284d8a902ec8d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:16.247131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbbd95998cd14b0f808284d8a902ec8d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:16.289149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bf8cf-ac3f-474f-b993-3beaf5b2f65a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:16.295907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350506a-65e8-492d-9b49-7739d4be4e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:16.296515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350506a-65e8-492d-9b49-7739d4be4e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:16.296788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350506a-65e8-492d-9b49-7739d4be4e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:16.297087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350506a-65e8-492d-9b49-7739d4be4e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:16.297799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350506a-65e8-492d-9b49-7739d4be4e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:16.299401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350506a-65e8-492d-9b49-7739d4be4e70 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:16.311098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1064/2089] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:34:16.315902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350506a-65e8-492d-9b49-7739d4be4e70 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:16.318757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7350506a-65e8-492d-9b49-7739d4be4e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:16.333144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350506a-65e8-492d-9b49-7739d4be4e70 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:16.338503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e64be3-5231-4365-b063-258840291fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:16.339233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e64be3-5231-4365-b063-258840291fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:16.339509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e64be3-5231-4365-b063-258840291fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:16.339782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e64be3-5231-4365-b063-258840291fdf admin admin] PATH_INFO: `/v3/projects/2cc7af711ad84c7bbcbc80989bf59c86/users/f339012f3b154a88b9c9e9a96ebfa88f/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:16.341814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e64be3-5231-4365-b063-258840291fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc7af711ad84c7bbcbc80989bf59c86, user_id=f339012f3b154a88b9c9e9a96ebfa88f, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:16.344576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e64be3-5231-4365-b063-258840291fdf admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:16.355522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1065/2090] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:34:16 2022] PUT /identity/v3/projects/2cc7af711ad84c7bbcbc80989bf59c86/users/f339012f3b154a88b9c9e9a96ebfa88f/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:34:16.363724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1026/2091] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:34:15 2022] DELETE /identity/v3/users/cbbd95998cd14b0f808284d8a902ec8d => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:34:16.366203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87879-8a89-45ad-8b9e-27a238bce495 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:16.366547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87879-8a89-45ad-8b9e-27a238bce495 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:16.366829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87879-8a89-45ad-8b9e-27a238bce495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:16.404206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-80d87879-8a89-45ad-8b9e-27a238bce495 None None] MFA Rules not processed for user `f339012f3b154a88b9c9e9a96ebfa88f`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:16.405450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-80d87879-8a89-45ad-8b9e-27a238bce495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:16.480911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1066/2092] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:34:16.492021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:16.493474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:16.553623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aabacd-59ba-4f1b-8bd4-2435e0897a83 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:16.557259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4423e-e441-4b56-881d-b619f79e4d7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:16.558517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4423e-e441-4b56-881d-b619f79e4d7a service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:16.558792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4423e-e441-4b56-881d-b619f79e4d7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:16.559058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4423e-e441-4b56-881d-b619f79e4d7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:16.568122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4423e-e441-4b56-881d-b619f79e4d7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:16.569586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4423e-e441-4b56-881d-b619f79e4d7a service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:16.582563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1027/2093] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:16.838387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87879-8a89-45ad-8b9e-27a238bce495 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:16.841035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-80d87879-8a89-45ad-8b9e-27a238bce495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:16.932851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87879-8a89-45ad-8b9e-27a238bce495 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:16.937826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:16.938469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:16.938750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:16.939016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] PATH_INFO: `/v3/projects/a7ac09397bbe43f6b80fc6e58228d9a9` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:16.940150 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7ac09397bbe43f6b80fc6e58228d9a9)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:16.942088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:16.968917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:17.211097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7ac09397bbe43f6b80fc6e58228d9a9', 'request_id': 'req-19b50d86-2746-428e-9b41-3dfa29d2115f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.218444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7ac09397bbe43f6b80fc6e58228d9a9', 'request_id': 'req-19b50d86-2746-428e-9b41-3dfa29d2115f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.344781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7ac09397bbe43f6b80fc6e58228d9a9', 'request_id': 'req-19b50d86-2746-428e-9b41-3dfa29d2115f'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.355752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] The token cache is being invalidate because project a7ac09397bbe43f6b80fc6e58228d9a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:34:17.356037 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.356742 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.358251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1067/2094] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:34:16 2022] DELETE /identity/v3/projects/a7ac09397bbe43f6b80fc6e58228d9a9 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:17.362286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4423e-e441-4b56-881d-b619f79e4d7a service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:17.373533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4423e-e441-4b56-881d-b619f79e4d7a service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:17.446269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:17.446269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:17.446269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:17.446768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] PATH_INFO: `/v3/users/d6b81e1ce1c84c9285192ca7fe38bef9` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:17.447511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b81e1ce1c84c9285192ca7fe38bef9)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:17.450120 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:17.577329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:17.732270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b81e1ce1c84c9285192ca7fe38bef9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.733608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b81e1ce1c84c9285192ca7fe38bef9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.734608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b81e1ce1c84c9285192ca7fe38bef9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.749545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b81e1ce1c84c9285192ca7fe38bef9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.774881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-086adb06-d181-49a2-a218-784e992ca909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b81e1ce1c84c9285192ca7fe38bef9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:17.927240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1028/2095] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:34:17 2022] DELETE /identity/v3/users/d6b81e1ce1c84c9285192ca7fe38bef9 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:18.594541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:18.596579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:18.616481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b50d86-2746-428e-9b41-3dfa29d2115f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:18.685517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:18.686157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:18.686434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:18.686707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] PATH_INFO: `/v3/projects/ad3e75b3082f462bb6b1435d8ee3623c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:18.687859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad3e75b3082f462bb6b1435d8ee3623c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:18.689817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:18.855531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad3e75b3082f462bb6b1435d8ee3623c', 'request_id': 'req-098f1e64-af18-4522-9a5a-3d61509ead2c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:18.862335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad3e75b3082f462bb6b1435d8ee3623c', 'request_id': 'req-098f1e64-af18-4522-9a5a-3d61509ead2c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:18.951182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad3e75b3082f462bb6b1435d8ee3623c', 'request_id': 'req-098f1e64-af18-4522-9a5a-3d61509ead2c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:18.970328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] The token cache is being invalidate because project ad3e75b3082f462bb6b1435d8ee3623c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:34:18.970938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:18.972055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-098f1e64-af18-4522-9a5a-3d61509ead2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:18.974120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1068/2096] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:34:18 2022] DELETE /identity/v3/projects/ad3e75b3082f462bb6b1435d8ee3623c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:18.979196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b79fe-5f7e-4573-9ffd-b2047497f0ac None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:18.980245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b79fe-5f7e-4573-9ffd-b2047497f0ac None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:18.980413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b79fe-5f7e-4573-9ffd-b2047497f0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:19.002066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-208b79fe-5f7e-4573-9ffd-b2047497f0ac None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:19.006164 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-208b79fe-5f7e-4573-9ffd-b2047497f0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:19.072364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1029/2097] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:34:19.080286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98275235-da02-41ca-9f41-6c7ac8d8df54 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:19.080597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98275235-da02-41ca-9f41-6c7ac8d8df54 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:19.080769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98275235-da02-41ca-9f41-6c7ac8d8df54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:19.101337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-98275235-da02-41ca-9f41-6c7ac8d8df54 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:19.102552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-98275235-da02-41ca-9f41-6c7ac8d8df54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:19.120917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1069/2098] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:34:19.125716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b79fe-5f7e-4573-9ffd-b2047497f0ac None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:19.138487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b79fe-5f7e-4573-9ffd-b2047497f0ac None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:19.143942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c36cb-eaaa-43c1-a4f8-79e57e987923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:19.145027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c36cb-eaaa-43c1-a4f8-79e57e987923 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:19.145467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c36cb-eaaa-43c1-a4f8-79e57e987923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:19.145806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c36cb-eaaa-43c1-a4f8-79e57e987923 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:19.146743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c36cb-eaaa-43c1-a4f8-79e57e987923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:34:19.147176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c36cb-eaaa-43c1-a4f8-79e57e987923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:19.149580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c36cb-eaaa-43c1-a4f8-79e57e987923 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:19.162493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1030/2099] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:34:19.168064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98275235-da02-41ca-9f41-6c7ac8d8df54 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:19.179490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98275235-da02-41ca-9f41-6c7ac8d8df54 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:19.185537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb2d51-28b0-4d49-ae1b-bb793856a6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:19.186173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb2d51-28b0-4d49-ae1b-bb793856a6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:19.186450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb2d51-28b0-4d49-ae1b-bb793856a6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:19.186717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb2d51-28b0-4d49-ae1b-bb793856a6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:19.187332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb2d51-28b0-4d49-ae1b-bb793856a6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:19.189418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb2d51-28b0-4d49-ae1b-bb793856a6f4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:19.357421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1070/2100] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:34:19 2022] POST /identity/v3/projects => generated 365 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:34:19.358718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c36cb-eaaa-43c1-a4f8-79e57e987923 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:19.369055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c36cb-eaaa-43c1-a4f8-79e57e987923 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:19.431002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6fcea-bda8-4130-9a7b-8ae78679ec58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:19.431551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6fcea-bda8-4130-9a7b-8ae78679ec58 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:19.431831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6fcea-bda8-4130-9a7b-8ae78679ec58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:19.432104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6fcea-bda8-4130-9a7b-8ae78679ec58 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:19.432919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6fcea-bda8-4130-9a7b-8ae78679ec58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:19.434965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6fcea-bda8-4130-9a7b-8ae78679ec58 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:19.559486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1031/2101] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:34:19 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:34:19.564903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb2d51-28b0-4d49-ae1b-bb793856a6f4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:19.579075 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb2d51-28b0-4d49-ae1b-bb793856a6f4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:19.584638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64d55e-c7db-43bc-b173-465ff5fdb887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:19.585557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d55e-c7db-43bc-b173-465ff5fdb887 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:19.585816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d55e-c7db-43bc-b173-465ff5fdb887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:19.586244 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64d55e-c7db-43bc-b173-465ff5fdb887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:19.587123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64d55e-c7db-43bc-b173-465ff5fdb887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:19.589929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64d55e-c7db-43bc-b173-465ff5fdb887 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:19.596515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:19.605211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1071/2102] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:34:19.609329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6fcea-bda8-4130-9a7b-8ae78679ec58 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:19.628152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6fcea-bda8-4130-9a7b-8ae78679ec58 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:19.636879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6624262-b211-46ed-9b31-0605f4af9c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:19.643278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6624262-b211-46ed-9b31-0605f4af9c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:19.643278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6624262-b211-46ed-9b31-0605f4af9c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:19.643278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6624262-b211-46ed-9b31-0605f4af9c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:19.643278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6624262-b211-46ed-9b31-0605f4af9c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:19.643278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6624262-b211-46ed-9b31-0605f4af9c0d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:19.670928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1032/2103] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:34:19.679223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64d55e-c7db-43bc-b173-465ff5fdb887 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:19.690509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64d55e-c7db-43bc-b173-465ff5fdb887 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:19.695305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da8b47-9328-4a29-8a81-9e5ca5401a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:19.695966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da8b47-9328-4a29-8a81-9e5ca5401a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:19.696237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da8b47-9328-4a29-8a81-9e5ca5401a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:19.696515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da8b47-9328-4a29-8a81-9e5ca5401a43 admin admin] PATH_INFO: `/v3/projects/191948ea02004177a488b2483cb1607b/users/46769294716e417dab410cb82d18c066/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:19.718898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da8b47-9328-4a29-8a81-9e5ca5401a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=191948ea02004177a488b2483cb1607b, user_id=46769294716e417dab410cb82d18c066, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:19.723209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da8b47-9328-4a29-8a81-9e5ca5401a43 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:19.851065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1072/2104] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:34:19 2022] PUT /identity/v3/projects/191948ea02004177a488b2483cb1607b/users/46769294716e417dab410cb82d18c066/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:19.864873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eac2aa-894e-4676-9140-d71d24c93e96 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:19.865212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eac2aa-894e-4676-9140-d71d24c93e96 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:19.865624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eac2aa-894e-4676-9140-d71d24c93e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:19.898205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:19.908049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-f3eac2aa-894e-4676-9140-d71d24c93e96 None None] MFA Rules not processed for user `46769294716e417dab410cb82d18c066`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:19.909825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f3eac2aa-894e-4676-9140-d71d24c93e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:19.997074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1033/2105] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:34:20.008906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da8b47-9328-4a29-8a81-9e5ca5401a43 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:20.011814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-52da8b47-9328-4a29-8a81-9e5ca5401a43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:20.086637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da8b47-9328-4a29-8a81-9e5ca5401a43 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:20.091789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30545347-169b-4057-97d7-c6e73a760e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:20.092430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30545347-169b-4057-97d7-c6e73a760e81 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:20.092681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30545347-169b-4057-97d7-c6e73a760e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:20.092989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30545347-169b-4057-97d7-c6e73a760e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:20.102022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30545347-169b-4057-97d7-c6e73a760e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:20.103455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30545347-169b-4057-97d7-c6e73a760e81 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:20.113480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1073/2106] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:34:21.459522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eac2aa-894e-4676-9140-d71d24c93e96 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:21.462771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f3eac2aa-894e-4676-9140-d71d24c93e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:21.543095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eac2aa-894e-4676-9140-d71d24c93e96 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:21.547583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:21.548219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:21.548491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:21.548820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] PATH_INFO: `/v3/users/133c7468b9e947498ab1e90de7b7b9e0` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:21.550141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=133c7468b9e947498ab1e90de7b7b9e0)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:21.552080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:21.789880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133c7468b9e947498ab1e90de7b7b9e0'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:21.790566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133c7468b9e947498ab1e90de7b7b9e0'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:21.791174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133c7468b9e947498ab1e90de7b7b9e0'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:21.800054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133c7468b9e947498ab1e90de7b7b9e0'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:21.812258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133c7468b9e947498ab1e90de7b7b9e0'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:21.973575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1034/2107] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:34:21 2022] DELETE /identity/v3/users/133c7468b9e947498ab1e90de7b7b9e0 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:21.978209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cf435-2dc6-42b5-ab40-7de1fa185561 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:21.978744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cf435-2dc6-42b5-ab40-7de1fa185561 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:21.978744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cf435-2dc6-42b5-ab40-7de1fa185561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:21.999291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e27cf435-2dc6-42b5-ab40-7de1fa185561 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:22.000768 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e27cf435-2dc6-42b5-ab40-7de1fa185561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:22.079093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1074/2108] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:34:22.089930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:22.092450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:22.120282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:22.168830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f0f70-ad7b-42e4-83f2-6985d38a5b3c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:22.173905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda1c4-a8b8-4974-9d4c-1a3390050b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:22.174795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda1c4-a8b8-4974-9d4c-1a3390050b05 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:22.175094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda1c4-a8b8-4974-9d4c-1a3390050b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:22.175372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda1c4-a8b8-4974-9d4c-1a3390050b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:22.192426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeda1c4-a8b8-4974-9d4c-1a3390050b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:22.193899 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeda1c4-a8b8-4974-9d4c-1a3390050b05 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:22.205107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1035/2109] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:22.823530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27cf435-2dc6-42b5-ab40-7de1fa185561 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:22.825529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e27cf435-2dc6-42b5-ab40-7de1fa185561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:22.843760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27cf435-2dc6-42b5-ab40-7de1fa185561 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:22.850039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:22.850758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:22.851098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:22.851431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] PATH_INFO: `/v3/projects/c6f81b64c0e54ba0b1122d884b2d703e` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:22.852770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f81b64c0e54ba0b1122d884b2d703e)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:22.855624 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:22.982416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f81b64c0e54ba0b1122d884b2d703e', 'request_id': 'req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:22.989825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f81b64c0e54ba0b1122d884b2d703e', 'request_id': 'req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:23.134255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f81b64c0e54ba0b1122d884b2d703e', 'request_id': 'req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:23.150127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] The token cache is being invalidate because project c6f81b64c0e54ba0b1122d884b2d703e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:34:23.150627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:23.152230 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf48cf03-1b41-4fc8-98d5-3c62f47adef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:23.155067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1075/2110] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:34:22 2022] DELETE /identity/v3/projects/c6f81b64c0e54ba0b1122d884b2d703e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:23.161568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda1c4-a8b8-4974-9d4c-1a3390050b05 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:23.184659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda1c4-a8b8-4974-9d4c-1a3390050b05 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:23.280369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:23.281045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:23.281326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:23.281951 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] PATH_INFO: `/v3/users/f339012f3b154a88b9c9e9a96ebfa88f` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:23.282761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f339012f3b154a88b9c9e9a96ebfa88f)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:23.284708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:23.342880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c055c-a476-4199-83af-3044e4614cbe None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:23.343166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c055c-a476-4199-83af-3044e4614cbe None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:23.343443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c055c-a476-4199-83af-3044e4614cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:23.351104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:23.373524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-120c055c-a476-4199-83af-3044e4614cbe None None] MFA Rules not processed for user `2d5bfffc316f481183093e386a326946`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:23.375086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-120c055c-a476-4199-83af-3044e4614cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:23.443537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1076/2111] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:34:23.603582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f339012f3b154a88b9c9e9a96ebfa88f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:23.604448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f339012f3b154a88b9c9e9a96ebfa88f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:23.604959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f339012f3b154a88b9c9e9a96ebfa88f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:23.614123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f339012f3b154a88b9c9e9a96ebfa88f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:23.632971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-57bd648e-580e-4130-a1fe-6a49cb22e63b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f339012f3b154a88b9c9e9a96ebfa88f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:23.707649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1036/2112] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:34:23 2022] DELETE /identity/v3/users/f339012f3b154a88b9c9e9a96ebfa88f => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:24.220742 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c055c-a476-4199-83af-3044e4614cbe None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:24.222799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-120c055c-a476-4199-83af-3044e4614cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:24.247636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c055c-a476-4199-83af-3044e4614cbe None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:24.343658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:24.344432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:24.344771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:24.349594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] PATH_INFO: `/v3/projects/2cc7af711ad84c7bbcbc80989bf59c86` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:24.349594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc7af711ad84c7bbcbc80989bf59c86)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:24.350813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:24.484061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc7af711ad84c7bbcbc80989bf59c86', 'request_id': 'req-c1c365ab-031e-408a-93d7-8c7e4b528c4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:24.490587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc7af711ad84c7bbcbc80989bf59c86', 'request_id': 'req-c1c365ab-031e-408a-93d7-8c7e4b528c4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:24.592324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc7af711ad84c7bbcbc80989bf59c86', 'request_id': 'req-c1c365ab-031e-408a-93d7-8c7e4b528c4c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:24.612941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] The token cache is being invalidate because project 2cc7af711ad84c7bbcbc80989bf59c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:34:24.613421 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:24.614499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c1c365ab-031e-408a-93d7-8c7e4b528c4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:34:24.617212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1077/2113] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:34:24 2022] DELETE /identity/v3/projects/2cc7af711ad84c7bbcbc80989bf59c86 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:24.626515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e135d-1e95-4d9c-8bfc-2bc27eeb9b6d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:24.627081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e135d-1e95-4d9c-8bfc-2bc27eeb9b6d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:24.628100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e135d-1e95-4d9c-8bfc-2bc27eeb9b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:24.652012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-3f8e135d-1e95-4d9c-8bfc-2bc27eeb9b6d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:24.654519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3f8e135d-1e95-4d9c-8bfc-2bc27eeb9b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:24.695981 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:24.724055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1037/2114] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:34:24.733343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297b77b-c303-4923-bb26-4eb7e70ee2b6 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:24.733782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297b77b-c303-4923-bb26-4eb7e70ee2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:24.734811 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297b77b-c303-4923-bb26-4eb7e70ee2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:24.757076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d297b77b-c303-4923-bb26-4eb7e70ee2b6 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:24.758596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d297b77b-c303-4923-bb26-4eb7e70ee2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:24.767189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1078/2115] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:34:24.771621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e135d-1e95-4d9c-8bfc-2bc27eeb9b6d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:24.783212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e135d-1e95-4d9c-8bfc-2bc27eeb9b6d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:24.787905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed910-9da0-417b-afc7-2125e302267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:24.788512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed910-9da0-417b-afc7-2125e302267a admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:24.788785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed910-9da0-417b-afc7-2125e302267a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:24.789159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ed910-9da0-417b-afc7-2125e302267a admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:24.789806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed910-9da0-417b-afc7-2125e302267a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:34:24.790102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed910-9da0-417b-afc7-2125e302267a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:24.792284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ed910-9da0-417b-afc7-2125e302267a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:24.804556 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1038/2116] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:34:24.810186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297b77b-c303-4923-bb26-4eb7e70ee2b6 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:24.819785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297b77b-c303-4923-bb26-4eb7e70ee2b6 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:24.825410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761666fe-3e0a-4c18-9bf3-97159e11df46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:24.826021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761666fe-3e0a-4c18-9bf3-97159e11df46 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:24.826292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761666fe-3e0a-4c18-9bf3-97159e11df46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:24.826560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761666fe-3e0a-4c18-9bf3-97159e11df46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:24.827222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761666fe-3e0a-4c18-9bf3-97159e11df46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:24.829777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761666fe-3e0a-4c18-9bf3-97159e11df46 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:24.924521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1079/2117] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:34:24 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:34:24.928895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed910-9da0-417b-afc7-2125e302267a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:24.941421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ed910-9da0-417b-afc7-2125e302267a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:25.003988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9809bd-f08e-402f-8eec-5e38a4305bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:25.004546 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9809bd-f08e-402f-8eec-5e38a4305bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:25.004849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9809bd-f08e-402f-8eec-5e38a4305bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:25.005126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9809bd-f08e-402f-8eec-5e38a4305bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:25.005754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9809bd-f08e-402f-8eec-5e38a4305bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:25.007680 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9809bd-f08e-402f-8eec-5e38a4305bd7 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:25.077325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1039/2118] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:34:24 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:34:25.082730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761666fe-3e0a-4c18-9bf3-97159e11df46 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:25.096198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761666fe-3e0a-4c18-9bf3-97159e11df46 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:25.100943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eea57a-258b-46c9-9135-d4cb10e9803f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:25.101559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eea57a-258b-46c9-9135-d4cb10e9803f admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:25.101837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eea57a-258b-46c9-9135-d4cb10e9803f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:25.102109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eea57a-258b-46c9-9135-d4cb10e9803f admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:25.102834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eea57a-258b-46c9-9135-d4cb10e9803f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:25.104567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eea57a-258b-46c9-9135-d4cb10e9803f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:25.108075 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:25.119489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1080/2119] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:34:25.124449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9809bd-f08e-402f-8eec-5e38a4305bd7 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:25.134864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9809bd-f08e-402f-8eec-5e38a4305bd7 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:25.140578 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98861fc-3b51-4426-ba83-b08c9d55538e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:25.141304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98861fc-3b51-4426-ba83-b08c9d55538e admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:25.141581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98861fc-3b51-4426-ba83-b08c9d55538e admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:25.141850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98861fc-3b51-4426-ba83-b08c9d55538e admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:25.142485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98861fc-3b51-4426-ba83-b08c9d55538e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:25.144140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98861fc-3b51-4426-ba83-b08c9d55538e admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:25.156087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1040/2120] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:34:25.160480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eea57a-258b-46c9-9135-d4cb10e9803f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:25.172184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eea57a-258b-46c9-9135-d4cb10e9803f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:25.176901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e91a71-d04f-4f1f-b811-6e770ffa9d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:25.177697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e91a71-d04f-4f1f-b811-6e770ffa9d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:25.177976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e91a71-d04f-4f1f-b811-6e770ffa9d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:25.178284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e91a71-d04f-4f1f-b811-6e770ffa9d13 admin admin] PATH_INFO: `/v3/projects/ae04996e51644750a28dc9029d067841/users/c1893ac3dc034bb29dfe9ee6733a239b/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:25.194711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e91a71-d04f-4f1f-b811-6e770ffa9d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae04996e51644750a28dc9029d067841, user_id=c1893ac3dc034bb29dfe9ee6733a239b, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:25.197372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e91a71-d04f-4f1f-b811-6e770ffa9d13 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:25.322472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1081/2121] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:34:25 2022] PUT /identity/v3/projects/ae04996e51644750a28dc9029d067841/users/c1893ac3dc034bb29dfe9ee6733a239b/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:25.334645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b5f3-3e09-4f66-a54d-b67c585cd6d1 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:25.334936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b5f3-3e09-4f66-a54d-b67c585cd6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:25.335203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b5f3-3e09-4f66-a54d-b67c585cd6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:25.373536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-7107b5f3-3e09-4f66-a54d-b67c585cd6d1 None None] MFA Rules not processed for user `c1893ac3dc034bb29dfe9ee6733a239b`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:25.374761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7107b5f3-3e09-4f66-a54d-b67c585cd6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:25.449447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1041/2122] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:34:25.455079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e91a71-d04f-4f1f-b811-6e770ffa9d13 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:25.527521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e91a71-d04f-4f1f-b811-6e770ffa9d13 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:25.532326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b775c-23a2-417a-b06b-8f9408b5910a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:25.532973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b775c-23a2-417a-b06b-8f9408b5910a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:25.533256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b775c-23a2-417a-b06b-8f9408b5910a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:25.533524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b775c-23a2-417a-b06b-8f9408b5910a admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:25.534147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b775c-23a2-417a-b06b-8f9408b5910a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:25.536062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b775c-23a2-417a-b06b-8f9408b5910a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:25.631511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1082/2123] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:34:25 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:34:25.636671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107b5f3-3e09-4f66-a54d-b67c585cd6d1 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:25.648309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107b5f3-3e09-4f66-a54d-b67c585cd6d1 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:25.690927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:25.715093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af03dc1-da43-4f46-b4a7-ad3acb46d548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:25.715700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af03dc1-da43-4f46-b4a7-ad3acb46d548 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:25.715978 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af03dc1-da43-4f46-b4a7-ad3acb46d548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:25.716248 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af03dc1-da43-4f46-b4a7-ad3acb46d548 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:25.716911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af03dc1-da43-4f46-b4a7-ad3acb46d548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:25.718948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af03dc1-da43-4f46-b4a7-ad3acb46d548 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:25.814650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1042/2124] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:34:25 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:34:25.818757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b775c-23a2-417a-b06b-8f9408b5910a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:25.829469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b775c-23a2-417a-b06b-8f9408b5910a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:25.834488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830eec4-3dcb-4467-8afa-b6d4dd383272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:25.835121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830eec4-3dcb-4467-8afa-b6d4dd383272 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:25.835403 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830eec4-3dcb-4467-8afa-b6d4dd383272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:25.835677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830eec4-3dcb-4467-8afa-b6d4dd383272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:25.836332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830eec4-3dcb-4467-8afa-b6d4dd383272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:25.837988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830eec4-3dcb-4467-8afa-b6d4dd383272 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:25.849929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1083/2125] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:34:25.854818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af03dc1-da43-4f46-b4a7-ad3acb46d548 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:25.865855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af03dc1-da43-4f46-b4a7-ad3acb46d548 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:25.870719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7a413-4ef4-4044-88a0-950e8f408097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:25.871381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7a413-4ef4-4044-88a0-950e8f408097 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:25.871664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7a413-4ef4-4044-88a0-950e8f408097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:25.871925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7a413-4ef4-4044-88a0-950e8f408097 admin admin] PATH_INFO: `/v3/projects/52d52a7b985c49bf88dc2225e0e8cb65/users/de36518c6e15437084a4d62f084a40c6/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:25.887847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7a413-4ef4-4044-88a0-950e8f408097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d52a7b985c49bf88dc2225e0e8cb65, user_id=de36518c6e15437084a4d62f084a40c6, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:25.890471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7a413-4ef4-4044-88a0-950e8f408097 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:26.021156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1043/2126] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:34:25 2022] PUT /identity/v3/projects/52d52a7b985c49bf88dc2225e0e8cb65/users/de36518c6e15437084a4d62f084a40c6/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:26.025692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830eec4-3dcb-4467-8afa-b6d4dd383272 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:26.036922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830eec4-3dcb-4467-8afa-b6d4dd383272 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:26.096362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa98e11-0a5a-4f6d-9ce3-ff7ea752e8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:26.096998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa98e11-0a5a-4f6d-9ce3-ff7ea752e8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:26.097277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa98e11-0a5a-4f6d-9ce3-ff7ea752e8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:26.097551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa98e11-0a5a-4f6d-9ce3-ff7ea752e8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:26.099946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa98e11-0a5a-4f6d-9ce3-ff7ea752e8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:26.099946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa98e11-0a5a-4f6d-9ce3-ff7ea752e8e9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:26.111774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1084/2127] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:34:26.116040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7a413-4ef4-4044-88a0-950e8f408097 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:26.126787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7a413-4ef4-4044-88a0-950e8f408097 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:26.131415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ae342-0097-4420-a82e-90afa52dd4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:26.132072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ae342-0097-4420-a82e-90afa52dd4d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:26.132348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ae342-0097-4420-a82e-90afa52dd4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:26.132618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ae342-0097-4420-a82e-90afa52dd4d0 admin admin] PATH_INFO: `/v3/projects/52d52a7b985c49bf88dc2225e0e8cb65/users/de36518c6e15437084a4d62f084a40c6/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:26.134587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ae342-0097-4420-a82e-90afa52dd4d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d52a7b985c49bf88dc2225e0e8cb65, user_id=de36518c6e15437084a4d62f084a40c6, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:26.137506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ae342-0097-4420-a82e-90afa52dd4d0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:26.147990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1044/2128] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:34:26 2022] PUT /identity/v3/projects/52d52a7b985c49bf88dc2225e0e8cb65/users/de36518c6e15437084a4d62f084a40c6/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:34:26.158618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d921-4ecf-45f7-a10a-cb92e9344b77 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:26.158901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d921-4ecf-45f7-a10a-cb92e9344b77 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:26.159201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07d921-4ecf-45f7-a10a-cb92e9344b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:26.189148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-db07d921-4ecf-45f7-a10a-cb92e9344b77 None None] MFA Rules not processed for user `de36518c6e15437084a4d62f084a40c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:34:26.190374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-db07d921-4ecf-45f7-a10a-cb92e9344b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:26.224969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:26.261002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1085/2129] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:34:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:34:26.271855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ae342-0097-4420-a82e-90afa52dd4d0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:26.273465 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-318ae342-0097-4420-a82e-90afa52dd4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:26.331640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ae342-0097-4420-a82e-90afa52dd4d0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:26.335436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e1bf6-8e6c-43c3-86c4-923c77003dfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:26.336038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1bf6-8e6c-43c3-86c4-923c77003dfb service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:26.336315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1bf6-8e6c-43c3-86c4-923c77003dfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:26.336584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e1bf6-8e6c-43c3-86c4-923c77003dfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:26.398571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e1bf6-8e6c-43c3-86c4-923c77003dfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:26.400147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e1bf6-8e6c-43c3-86c4-923c77003dfb service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:26.409953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1045/2130] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:36.333462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=14 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:36.405847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:40.913126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07d921-4ecf-45f7-a10a-cb92e9344b77 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:40.913126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-db07d921-4ecf-45f7-a10a-cb92e9344b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:41.002961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07d921-4ecf-45f7-a10a-cb92e9344b77 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:41.008928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609ca43-47fa-43b7-96d2-cf39415c732f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:41.009739 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609ca43-47fa-43b7-96d2-cf39415c732f service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:41.010022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609ca43-47fa-43b7-96d2-cf39415c732f service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:41.010294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609ca43-47fa-43b7-96d2-cf39415c732f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:41.012295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6609ca43-47fa-43b7-96d2-cf39415c732f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:34:41.029066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609ca43-47fa-43b7-96d2-cf39415c732f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:41.031494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609ca43-47fa-43b7-96d2-cf39415c732f service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:41.043526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1086/2131] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:41.350804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e1bf6-8e6c-43c3-86c4-923c77003dfb service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:34:41.357171 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e1bf6-8e6c-43c3-86c4-923c77003dfb service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:34:41.361716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547daf2-23d3-49f1-91d7-8100a3b6b7cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:34:41.362416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547daf2-23d3-49f1-91d7-8100a3b6b7cd service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:34:41.362745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547daf2-23d3-49f1-91d7-8100a3b6b7cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:34:41.363016 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547daf2-23d3-49f1-91d7-8100a3b6b7cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:34:41.374250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547daf2-23d3-49f1-91d7-8100a3b6b7cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:34:41.375724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547daf2-23d3-49f1-91d7-8100a3b6b7cd service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:34:41.387493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1046/2132] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:34:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:34:51.038060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=32 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:34:51.381973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=6 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:48.403739 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609ca43-47fa-43b7-96d2-cf39415c732f service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:48.447048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53da9db-c6ab-46b4-93dd-e3b9c3e42e24 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:48.447757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53da9db-c6ab-46b4-93dd-e3b9c3e42e24 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:48.447757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53da9db-c6ab-46b4-93dd-e3b9c3e42e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:48.468071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c53da9db-c6ab-46b4-93dd-e3b9c3e42e24 None None] MFA Rules not processed for user `5055ddeab8314c93bb2a664f9d1a7f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:48.469322 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c53da9db-c6ab-46b4-93dd-e3b9c3e42e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:48.480540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609ca43-47fa-43b7-96d2-cf39415c732f service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:48.485809 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:48.486711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:48.487142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:48.487552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] PATH_INFO: `/v3/users/c1893ac3dc034bb29dfe9ee6733a239b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:48.489656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1893ac3dc034bb29dfe9ee6733a239b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:48.493501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:48.537020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1047/2133] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:35:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:35:48.808623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1893ac3dc034bb29dfe9ee6733a239b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:48.931478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1893ac3dc034bb29dfe9ee6733a239b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:48.949596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1893ac3dc034bb29dfe9ee6733a239b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:48.950556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1893ac3dc034bb29dfe9ee6733a239b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:48.961116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1893ac3dc034bb29dfe9ee6733a239b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:48.968274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1087/2134] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:35:48 2022] DELETE /identity/v3/users/c1893ac3dc034bb29dfe9ee6733a239b => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:48.973844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca942-456e-4c77-861f-2f30c79c7ba0 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:48.974223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca942-456e-4c77-861f-2f30c79c7ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:48.974466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca942-456e-4c77-861f-2f30c79c7ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:48.999662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-dd4ca942-456e-4c77-861f-2f30c79c7ba0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:49.000940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-dd4ca942-456e-4c77-861f-2f30c79c7ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:49.091410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1048/2135] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:35:49.105364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:49.106940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:49.187417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4108b-dbbc-4386-afa3-a21adc6ff68a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:49.192740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5405-6c02-495e-baa2-a7a66bc31619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:49.193658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5405-6c02-495e-baa2-a7a66bc31619 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:49.194146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5405-6c02-495e-baa2-a7a66bc31619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:49.194528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a5405-6c02-495e-baa2-a7a66bc31619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:49.206545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a5405-6c02-495e-baa2-a7a66bc31619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:49.209262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a5405-6c02-495e-baa2-a7a66bc31619 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:49.220955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1088/2136] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:35:49.935031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca942-456e-4c77-861f-2f30c79c7ba0 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:49.937821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-dd4ca942-456e-4c77-861f-2f30c79c7ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:49.953513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca942-456e-4c77-861f-2f30c79c7ba0 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:49.960743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:49.961473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:49.961753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:49.962029 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] PATH_INFO: `/v3/projects/ae04996e51644750a28dc9029d067841` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:49.963419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae04996e51644750a28dc9029d067841)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:49.966330 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:50.180190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae04996e51644750a28dc9029d067841', 'request_id': 'req-184f9945-f212-4847-bfe6-d439b508b3c5'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:50.189139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae04996e51644750a28dc9029d067841', 'request_id': 'req-184f9945-f212-4847-bfe6-d439b508b3c5'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:50.378059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae04996e51644750a28dc9029d067841', 'request_id': 'req-184f9945-f212-4847-bfe6-d439b508b3c5'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:50.396252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] The token cache is being invalidate because project ae04996e51644750a28dc9029d067841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:35:50.396581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:50.397916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:50.399849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1049/2137] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:35:49 2022] DELETE /identity/v3/projects/ae04996e51644750a28dc9029d067841 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:50.404699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5405-6c02-495e-baa2-a7a66bc31619 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:50.421694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a5405-6c02-495e-baa2-a7a66bc31619 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:50.447706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:50.517270 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:50.518374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:50.518374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:50.518716 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] PATH_INFO: `/v3/users/de36518c6e15437084a4d62f084a40c6` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:50.519938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de36518c6e15437084a4d62f084a40c6)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:50.523621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:50.926045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de36518c6e15437084a4d62f084a40c6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:51.195446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de36518c6e15437084a4d62f084a40c6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:51.215431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de36518c6e15437084a4d62f084a40c6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:51.216326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de36518c6e15437084a4d62f084a40c6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:51.225038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-0cb40439-3418-4007-a0d5-ee7d150bdea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de36518c6e15437084a4d62f084a40c6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:51.233001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1089/2138] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:35:50 2022] DELETE /identity/v3/users/de36518c6e15437084a4d62f084a40c6 => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:51.940221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:51.952745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:51.960576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f9945-f212-4847-bfe6-d439b508b3c5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:51.993226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:52.043725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:52.044480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:52.044814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:52.045190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] PATH_INFO: `/v3/projects/52d52a7b985c49bf88dc2225e0e8cb65` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:52.046494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d52a7b985c49bf88dc2225e0e8cb65)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:52.048531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:52.308192 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d52a7b985c49bf88dc2225e0e8cb65', 'request_id': 'req-aedab285-e798-48f9-b7f9-96e1f04598eb'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:52.319496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d52a7b985c49bf88dc2225e0e8cb65', 'request_id': 'req-aedab285-e798-48f9-b7f9-96e1f04598eb'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:52.525560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d52a7b985c49bf88dc2225e0e8cb65', 'request_id': 'req-aedab285-e798-48f9-b7f9-96e1f04598eb'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:52.541094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] The token cache is being invalidate because project 52d52a7b985c49bf88dc2225e0e8cb65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:35:52.541407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:52.542206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aedab285-e798-48f9-b7f9-96e1f04598eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:52.543885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1050/2139] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:35:51 2022] DELETE /identity/v3/projects/52d52a7b985c49bf88dc2225e0e8cb65 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:52.551159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680adcf0-e547-420b-aefb-de986fa53be8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:52.552246 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680adcf0-e547-420b-aefb-de986fa53be8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:52.552246 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680adcf0-e547-420b-aefb-de986fa53be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:52.578380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-680adcf0-e547-420b-aefb-de986fa53be8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:52.579498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-680adcf0-e547-420b-aefb-de986fa53be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:52.681356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1090/2140] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:52.689778 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd570e-e242-4dd1-80a5-0bf78638f4da None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:52.690412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd570e-e242-4dd1-80a5-0bf78638f4da None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:52.691154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd570e-e242-4dd1-80a5-0bf78638f4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:52.718618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6cdd570e-e242-4dd1-80a5-0bf78638f4da None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:52.719953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6cdd570e-e242-4dd1-80a5-0bf78638f4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:52.729016 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1051/2141] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:52.754747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680adcf0-e547-420b-aefb-de986fa53be8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:52.766036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680adcf0-e547-420b-aefb-de986fa53be8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:52.771602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b87124-a1be-4188-afb1-79e81bf12ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:52.773048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b87124-a1be-4188-afb1-79e81bf12ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:52.773515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b87124-a1be-4188-afb1-79e81bf12ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:52.773826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b87124-a1be-4188-afb1-79e81bf12ab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:52.774514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b87124-a1be-4188-afb1-79e81bf12ab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:35:52.774816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b87124-a1be-4188-afb1-79e81bf12ab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:52.776467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b87124-a1be-4188-afb1-79e81bf12ab3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:52.796314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1091/2142] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:35:52.801596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd570e-e242-4dd1-80a5-0bf78638f4da None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:52.813913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd570e-e242-4dd1-80a5-0bf78638f4da None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:52.821885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9afe618-4aad-4fde-accb-8f3006244ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:52.822518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afe618-4aad-4fde-accb-8f3006244ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:52.822797 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afe618-4aad-4fde-accb-8f3006244ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:52.823094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9afe618-4aad-4fde-accb-8f3006244ea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:52.823781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9afe618-4aad-4fde-accb-8f3006244ea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:52.825760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9afe618-4aad-4fde-accb-8f3006244ea3 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:53.026979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1052/2143] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:35:52 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:35:53.031562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b87124-a1be-4188-afb1-79e81bf12ab3 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:53.044249 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b87124-a1be-4188-afb1-79e81bf12ab3 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:53.073238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:53.123550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94aa9f5-6980-42f8-bba9-9c3fadf9b7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:53.124378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94aa9f5-6980-42f8-bba9-9c3fadf9b7eb admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:53.124730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94aa9f5-6980-42f8-bba9-9c3fadf9b7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:53.125045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94aa9f5-6980-42f8-bba9-9c3fadf9b7eb admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:53.125862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94aa9f5-6980-42f8-bba9-9c3fadf9b7eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:53.128164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94aa9f5-6980-42f8-bba9-9c3fadf9b7eb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:53.282083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1092/2144] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:35:53 2022] POST /identity/v3/users => generated 294 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:35:53.287458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9afe618-4aad-4fde-accb-8f3006244ea3 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:53.300689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9afe618-4aad-4fde-accb-8f3006244ea3 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:53.309369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9668f6-977c-4fc4-93d3-4c0d5ff70f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:53.310437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9668f6-977c-4fc4-93d3-4c0d5ff70f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:53.310914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9668f6-977c-4fc4-93d3-4c0d5ff70f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:53.311256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9668f6-977c-4fc4-93d3-4c0d5ff70f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:53.312007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9668f6-977c-4fc4-93d3-4c0d5ff70f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:53.313879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9668f6-977c-4fc4-93d3-4c0d5ff70f7b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:53.336860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1053/2145] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:35:53.341752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94aa9f5-6980-42f8-bba9-9c3fadf9b7eb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:53.353603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94aa9f5-6980-42f8-bba9-9c3fadf9b7eb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:53.360770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7a2a1-5a12-4289-b80f-3e45e11fa371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:53.361634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7a2a1-5a12-4289-b80f-3e45e11fa371 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:53.361970 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7a2a1-5a12-4289-b80f-3e45e11fa371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:53.362294 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7a2a1-5a12-4289-b80f-3e45e11fa371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:53.363091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7a2a1-5a12-4289-b80f-3e45e11fa371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:53.364928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7a2a1-5a12-4289-b80f-3e45e11fa371 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:53.378419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1093/2146] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:35:53.385306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9668f6-977c-4fc4-93d3-4c0d5ff70f7b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:53.399351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9668f6-977c-4fc4-93d3-4c0d5ff70f7b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:53.405529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd21bd4-3215-4ede-abdd-e3788f4d82a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:53.406222 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21bd4-3215-4ede-abdd-e3788f4d82a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:53.406495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21bd4-3215-4ede-abdd-e3788f4d82a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:53.406789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd21bd4-3215-4ede-abdd-e3788f4d82a5 admin admin] PATH_INFO: `/v3/projects/49156af4c98d48ef9d38a4e05b07573c/users/ea3bf37937f14a6ab2c6f2c394b07f5d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:53.426955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd21bd4-3215-4ede-abdd-e3788f4d82a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49156af4c98d48ef9d38a4e05b07573c, user_id=ea3bf37937f14a6ab2c6f2c394b07f5d, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:53.430671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd21bd4-3215-4ede-abdd-e3788f4d82a5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:53.603579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1054/2147] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:35:53 2022] PUT /identity/v3/projects/49156af4c98d48ef9d38a4e05b07573c/users/ea3bf37937f14a6ab2c6f2c394b07f5d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:35:53.614883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d77c2-b051-48f5-96ba-5b97114295ed None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:53.615530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d77c2-b051-48f5-96ba-5b97114295ed None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:53.616529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d77c2-b051-48f5-96ba-5b97114295ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:53.659890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-3e7d77c2-b051-48f5-96ba-5b97114295ed None None] MFA Rules not processed for user `ea3bf37937f14a6ab2c6f2c394b07f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:53.661890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3e7d77c2-b051-48f5-96ba-5b97114295ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:53.748193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1094/2148] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:35:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:35:53.754716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd21bd4-3215-4ede-abdd-e3788f4d82a5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:53.855592 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:53.863991 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd21bd4-3215-4ede-abdd-e3788f4d82a5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:53.872046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660130e-b628-4007-9f94-aad785771adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:53.873622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660130e-b628-4007-9f94-aad785771adb admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:53.873622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660130e-b628-4007-9f94-aad785771adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:53.873622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660130e-b628-4007-9f94-aad785771adb admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:53.874032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660130e-b628-4007-9f94-aad785771adb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:53.875832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660130e-b628-4007-9f94-aad785771adb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:54.062630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1055/2149] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:35:53 2022] POST /identity/v3/projects => generated 387 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:35:54.067972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d77c2-b051-48f5-96ba-5b97114295ed None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:54.081493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d77c2-b051-48f5-96ba-5b97114295ed None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:54.156927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9644e8-3ac3-4c67-b03f-e9f42b688548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:54.157552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9644e8-3ac3-4c67-b03f-e9f42b688548 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:54.157880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9644e8-3ac3-4c67-b03f-e9f42b688548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:54.158298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9644e8-3ac3-4c67-b03f-e9f42b688548 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:54.158969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9644e8-3ac3-4c67-b03f-e9f42b688548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:54.161093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9644e8-3ac3-4c67-b03f-e9f42b688548 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:54.355732 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1095/2150] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:35:54 2022] POST /identity/v3/users => generated 300 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:35:54.364881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660130e-b628-4007-9f94-aad785771adb admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:54.377045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660130e-b628-4007-9f94-aad785771adb admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:54.384957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4680b-836e-49ab-aba9-e5bbf188215d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:54.389608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4680b-836e-49ab-aba9-e5bbf188215d admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:54.389608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4680b-836e-49ab-aba9-e5bbf188215d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:54.389608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4680b-836e-49ab-aba9-e5bbf188215d admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:54.389608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4680b-836e-49ab-aba9-e5bbf188215d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:54.390016 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4680b-836e-49ab-aba9-e5bbf188215d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:54.405934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1056/2151] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:35:54.411852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9644e8-3ac3-4c67-b03f-e9f42b688548 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:54.425470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9644e8-3ac3-4c67-b03f-e9f42b688548 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:54.437814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939d2d1-259d-4efd-9c92-824a245f93f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:54.438302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939d2d1-259d-4efd-9c92-824a245f93f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:54.447190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939d2d1-259d-4efd-9c92-824a245f93f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:54.447190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939d2d1-259d-4efd-9c92-824a245f93f6 admin admin] PATH_INFO: `/v3/projects/bfa2a79a10ee4452bb588b2e934af24a/users/30ceecbe747c4ea3b0d54c8f511ee62b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:54.469261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939d2d1-259d-4efd-9c92-824a245f93f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa2a79a10ee4452bb588b2e934af24a, user_id=30ceecbe747c4ea3b0d54c8f511ee62b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:54.472059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939d2d1-259d-4efd-9c92-824a245f93f6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:54.614633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1096/2152] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:35:54 2022] PUT /identity/v3/projects/bfa2a79a10ee4452bb588b2e934af24a/users/30ceecbe747c4ea3b0d54c8f511ee62b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:54.619793 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4680b-836e-49ab-aba9-e5bbf188215d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:54.633254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4680b-836e-49ab-aba9-e5bbf188215d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:54.732673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea75bd-0636-4215-9c31-b6eb76c35ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:54.733328 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea75bd-0636-4215-9c31-b6eb76c35ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:54.733610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea75bd-0636-4215-9c31-b6eb76c35ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:54.733883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea75bd-0636-4215-9c31-b6eb76c35ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:54.734568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea75bd-0636-4215-9c31-b6eb76c35ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:54.736239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea75bd-0636-4215-9c31-b6eb76c35ad8 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:54.756905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1057/2153] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:35:54.758547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939d2d1-259d-4efd-9c92-824a245f93f6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:54.765521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:54.772526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939d2d1-259d-4efd-9c92-824a245f93f6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:54.788411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72969966-78b8-48d9-81a6-44624371d4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:54.790199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72969966-78b8-48d9-81a6-44624371d4d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:54.797797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72969966-78b8-48d9-81a6-44624371d4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:54.797797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72969966-78b8-48d9-81a6-44624371d4d8 admin admin] PATH_INFO: `/v3/projects/bfa2a79a10ee4452bb588b2e934af24a/users/30ceecbe747c4ea3b0d54c8f511ee62b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:54.797797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72969966-78b8-48d9-81a6-44624371d4d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa2a79a10ee4452bb588b2e934af24a, user_id=30ceecbe747c4ea3b0d54c8f511ee62b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:54.797797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72969966-78b8-48d9-81a6-44624371d4d8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:54.815587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1097/2154] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:35:54 2022] PUT /identity/v3/projects/bfa2a79a10ee4452bb588b2e934af24a/users/30ceecbe747c4ea3b0d54c8f511ee62b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:54.827123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0993000-f154-470d-b16a-9b22c45a5db2 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:54.828167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0993000-f154-470d-b16a-9b22c45a5db2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:54.828167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0993000-f154-470d-b16a-9b22c45a5db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:54.872025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e0993000-f154-470d-b16a-9b22c45a5db2 None None] MFA Rules not processed for user `30ceecbe747c4ea3b0d54c8f511ee62b`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:54.873990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e0993000-f154-470d-b16a-9b22c45a5db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:55.010844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1058/2155] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:35:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:55.029274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72969966-78b8-48d9-81a6-44624371d4d8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:55.040189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-72969966-78b8-48d9-81a6-44624371d4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:55.131461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72969966-78b8-48d9-81a6-44624371d4d8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:55.137035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e56847-1fbc-44d3-89d6-a81cfcffb90d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:55.137685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e56847-1fbc-44d3-89d6-a81cfcffb90d service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:55.137967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e56847-1fbc-44d3-89d6-a81cfcffb90d service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:55.138243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e56847-1fbc-44d3-89d6-a81cfcffb90d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:55.164547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e56847-1fbc-44d3-89d6-a81cfcffb90d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:55.166276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e56847-1fbc-44d3-89d6-a81cfcffb90d service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:55.181475 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1098/2156] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:35:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:35:55.234913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0993000-f154-470d-b16a-9b22c45a5db2 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:55.292910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:55.306101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0993000-f154-470d-b16a-9b22c45a5db2 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:55.312935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:55.313725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:55.313998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:55.314271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] PATH_INFO: `/v3/users/ea3bf37937f14a6ab2c6f2c394b07f5d` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:55.315461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3bf37937f14a6ab2c6f2c394b07f5d)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:55.317567 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:55.727302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3bf37937f14a6ab2c6f2c394b07f5d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:55.727968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3bf37937f14a6ab2c6f2c394b07f5d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:55.728703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3bf37937f14a6ab2c6f2c394b07f5d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:55.737027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3bf37937f14a6ab2c6f2c394b07f5d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:55.750841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3bf37937f14a6ab2c6f2c394b07f5d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:55.978857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1059/2157] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:35:55 2022] DELETE /identity/v3/users/ea3bf37937f14a6ab2c6f2c394b07f5d => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:55.984001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f8883-16b1-4b65-b3fa-c855540e2e79 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:55.984601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f8883-16b1-4b65-b3fa-c855540e2e79 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:55.984601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f8883-16b1-4b65-b3fa-c855540e2e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:56.012239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-449f8883-16b1-4b65-b3fa-c855540e2e79 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:56.013948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-449f8883-16b1-4b65-b3fa-c855540e2e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:56.128929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1099/2158] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:56.138371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:56.139951 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:56.236252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ddad1-8691-4d1b-933d-c18954a78749 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:56.239994 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2a9a8-50e7-4e78-9e83-f6176bf57517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:56.240653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2a9a8-50e7-4e78-9e83-f6176bf57517 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:56.241585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2a9a8-50e7-4e78-9e83-f6176bf57517 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:56.241861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2a9a8-50e7-4e78-9e83-f6176bf57517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:56.260686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2a9a8-50e7-4e78-9e83-f6176bf57517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:56.262939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2a9a8-50e7-4e78-9e83-f6176bf57517 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:56.275320 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1060/2159] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:35:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:35:57.299237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f8883-16b1-4b65-b3fa-c855540e2e79 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:57.302363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-449f8883-16b1-4b65-b3fa-c855540e2e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:57.317451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f8883-16b1-4b65-b3fa-c855540e2e79 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:57.328172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:57.329222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:57.329717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:57.330190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] PATH_INFO: `/v3/projects/49156af4c98d48ef9d38a4e05b07573c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:57.331844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49156af4c98d48ef9d38a4e05b07573c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:57.335631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:57.554803 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49156af4c98d48ef9d38a4e05b07573c', 'request_id': 'req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:57.561111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49156af4c98d48ef9d38a4e05b07573c', 'request_id': 'req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:57.708438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49156af4c98d48ef9d38a4e05b07573c', 'request_id': 'req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:57.724196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] The token cache is being invalidate because project 49156af4c98d48ef9d38a4e05b07573c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:35:57.725333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:57.727059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:57.729258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1100/2160] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:35:57 2022] DELETE /identity/v3/projects/49156af4c98d48ef9d38a4e05b07573c => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:57.733298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2a9a8-50e7-4e78-9e83-f6176bf57517 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:57.744604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2a9a8-50e7-4e78-9e83-f6176bf57517 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:57.824020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:57.824683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:57.824997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:57.825270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] PATH_INFO: `/v3/users/30ceecbe747c4ea3b0d54c8f511ee62b` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:57.826425 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ceecbe747c4ea3b0d54c8f511ee62b)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:57.828377 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:57.843883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:58.242038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ceecbe747c4ea3b0d54c8f511ee62b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:58.243318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ceecbe747c4ea3b0d54c8f511ee62b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:58.243318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ceecbe747c4ea3b0d54c8f511ee62b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:58.250545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ceecbe747c4ea3b0d54c8f511ee62b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:58.262774 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3c91f87c-20e7-4cda-ae35-2337682bd037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ceecbe747c4ea3b0d54c8f511ee62b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:58.406148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1061/2161] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:35:57 2022] DELETE /identity/v3/users/30ceecbe747c4ea3b0d54c8f511ee62b => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:59.141226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:59.143815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:59.161558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b3e0cc-b01f-49ea-b2f4-2fc3f9817da8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:59.179437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:35:59.227044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:59.227680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:59.227954 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:59.228221 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] PATH_INFO: `/v3/projects/bfa2a79a10ee4452bb588b2e934af24a` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:59.229360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa2a79a10ee4452bb588b2e934af24a)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:59.231259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:59.423734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa2a79a10ee4452bb588b2e934af24a', 'request_id': 'req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:59.430981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa2a79a10ee4452bb588b2e934af24a', 'request_id': 'req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:59.599433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa2a79a10ee4452bb588b2e934af24a', 'request_id': 'req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:59.611276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] The token cache is being invalidate because project bfa2a79a10ee4452bb588b2e934af24a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:35:59.611583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:59.612241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-bbe0ac1b-91cd-414c-94b1-f720f3ae26ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:35:59.613751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1101/2162] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:35:59 2022] DELETE /identity/v3/projects/bfa2a79a10ee4452bb588b2e934af24a => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:35:59.619703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970cea-b127-499f-a16f-63e6b914e6aa None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:59.621384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970cea-b127-499f-a16f-63e6b914e6aa None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:59.621907 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970cea-b127-499f-a16f-63e6b914e6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:59.641839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-44970cea-b127-499f-a16f-63e6b914e6aa None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:59.643041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-44970cea-b127-499f-a16f-63e6b914e6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:59.712281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1062/2163] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:59.720184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cd451-dc05-47c0-9126-1fb0e561c9e9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:59.720474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cd451-dc05-47c0-9126-1fb0e561c9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:59.720751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cd451-dc05-47c0-9126-1fb0e561c9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:59.743171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-756cd451-dc05-47c0-9126-1fb0e561c9e9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:35:59.744549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-756cd451-dc05-47c0-9126-1fb0e561c9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:35:59.753159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1102/2164] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:35:59.757806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970cea-b127-499f-a16f-63e6b914e6aa None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:59.768899 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970cea-b127-499f-a16f-63e6b914e6aa None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:59.774845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59955a54-da20-428e-8f44-3c767d276d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:59.775471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59955a54-da20-428e-8f44-3c767d276d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:59.775748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59955a54-da20-428e-8f44-3c767d276d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:59.776017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59955a54-da20-428e-8f44-3c767d276d8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:59.776666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59955a54-da20-428e-8f44-3c767d276d8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:35:59.777015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59955a54-da20-428e-8f44-3c767d276d8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:59.778621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59955a54-da20-428e-8f44-3c767d276d8f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:59.791725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1063/2165] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:35:59.796198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cd451-dc05-47c0-9126-1fb0e561c9e9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:35:59.805986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cd451-dc05-47c0-9126-1fb0e561c9e9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:35:59.810857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f5046-e8fb-48bb-a114-4f5a8e299991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:35:59.811512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5046-e8fb-48bb-a114-4f5a8e299991 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:35:59.811880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5046-e8fb-48bb-a114-4f5a8e299991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:35:59.812153 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5046-e8fb-48bb-a114-4f5a8e299991 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:35:59.812791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f5046-e8fb-48bb-a114-4f5a8e299991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:35:59.814701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f5046-e8fb-48bb-a114-4f5a8e299991 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:35:59.990520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1103/2166] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:35:59 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:35:59.995318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59955a54-da20-428e-8f44-3c767d276d8f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:00.007157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59955a54-da20-428e-8f44-3c767d276d8f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:00.080977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c16695-c782-421b-a201-b04dff30ae03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:00.081606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c16695-c782-421b-a201-b04dff30ae03 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:00.084745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c16695-c782-421b-a201-b04dff30ae03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:00.084745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c16695-c782-421b-a201-b04dff30ae03 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:00.084745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c16695-c782-421b-a201-b04dff30ae03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:00.085546 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c16695-c782-421b-a201-b04dff30ae03 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:00.241679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1064/2167] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:35:59 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:00.246623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f5046-e8fb-48bb-a114-4f5a8e299991 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:00.258605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f5046-e8fb-48bb-a114-4f5a8e299991 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:00.263541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634438f-b703-4c50-8451-300873bf4f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:00.264297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634438f-b703-4c50-8451-300873bf4f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:00.264580 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634438f-b703-4c50-8451-300873bf4f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:00.264891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634438f-b703-4c50-8451-300873bf4f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:00.265539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634438f-b703-4c50-8451-300873bf4f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:00.267259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634438f-b703-4c50-8451-300873bf4f95 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:00.280295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1104/2168] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:00.284767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c16695-c782-421b-a201-b04dff30ae03 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:00.295477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c16695-c782-421b-a201-b04dff30ae03 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:00.300455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262293ee-4a61-41e4-932e-ef61c4863856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:00.301138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262293ee-4a61-41e4-932e-ef61c4863856 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:00.301458 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262293ee-4a61-41e4-932e-ef61c4863856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:00.301777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262293ee-4a61-41e4-932e-ef61c4863856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:00.302388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262293ee-4a61-41e4-932e-ef61c4863856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:00.304837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262293ee-4a61-41e4-932e-ef61c4863856 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:00.307445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:00.317119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1065/2169] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:00.323368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634438f-b703-4c50-8451-300873bf4f95 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:00.336631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634438f-b703-4c50-8451-300873bf4f95 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:00.341318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8c267-6bac-4088-8c05-4f6c0cda538c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:00.341979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8c267-6bac-4088-8c05-4f6c0cda538c admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:00.342290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8c267-6bac-4088-8c05-4f6c0cda538c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:00.342578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8c267-6bac-4088-8c05-4f6c0cda538c admin admin] PATH_INFO: `/v3/projects/8f59f8adebfc4ad8aea318ae32ce930e/users/337b7cd9c6fa47d3b176ea6389764f7b/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:00.359048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8c267-6bac-4088-8c05-4f6c0cda538c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f59f8adebfc4ad8aea318ae32ce930e, user_id=337b7cd9c6fa47d3b176ea6389764f7b, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:00.361820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8c267-6bac-4088-8c05-4f6c0cda538c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:00.600253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1105/2170] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:00 2022] PUT /identity/v3/projects/8f59f8adebfc4ad8aea318ae32ce930e/users/337b7cd9c6fa47d3b176ea6389764f7b/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:00.611057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a991c99-b40b-47f2-af3b-cba79c7e1c81 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:00.611433 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a991c99-b40b-47f2-af3b-cba79c7e1c81 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:00.611715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a991c99-b40b-47f2-af3b-cba79c7e1c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:00.641175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9a991c99-b40b-47f2-af3b-cba79c7e1c81 None None] MFA Rules not processed for user `337b7cd9c6fa47d3b176ea6389764f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:00.642408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9a991c99-b40b-47f2-af3b-cba79c7e1c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:00.713757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1066/2171] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:00.718494 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8c267-6bac-4088-8c05-4f6c0cda538c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:00.766530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:00.785284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8c267-6bac-4088-8c05-4f6c0cda538c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:00.791198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816deea-7241-4746-9338-9bfd214c2ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:00.802323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816deea-7241-4746-9338-9bfd214c2ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:00.802323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816deea-7241-4746-9338-9bfd214c2ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:00.802323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816deea-7241-4746-9338-9bfd214c2ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:00.802323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816deea-7241-4746-9338-9bfd214c2ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:00.802323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816deea-7241-4746-9338-9bfd214c2ad6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:00.939767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1106/2172] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:00 2022] POST /identity/v3/projects => generated 379 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:00.943963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a991c99-b40b-47f2-af3b-cba79c7e1c81 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:00.955419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a991c99-b40b-47f2-af3b-cba79c7e1c81 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:01.043768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cd681-e68c-4103-b100-5908e077de33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:01.044553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd681-e68c-4103-b100-5908e077de33 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:01.045033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd681-e68c-4103-b100-5908e077de33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:01.045404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd681-e68c-4103-b100-5908e077de33 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:01.046177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cd681-e68c-4103-b100-5908e077de33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:01.048631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cd681-e68c-4103-b100-5908e077de33 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:01.160061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1067/2173] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:00 2022] POST /identity/v3/users => generated 296 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:01.164962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816deea-7241-4746-9338-9bfd214c2ad6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:01.180740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816deea-7241-4746-9338-9bfd214c2ad6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:01.188987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd63d5f-929b-47e4-9a4b-e4416f537ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:01.190016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd63d5f-929b-47e4-9a4b-e4416f537ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:01.190389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd63d5f-929b-47e4-9a4b-e4416f537ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:01.190802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd63d5f-929b-47e4-9a4b-e4416f537ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:01.191888 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd63d5f-929b-47e4-9a4b-e4416f537ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:01.194890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd63d5f-929b-47e4-9a4b-e4416f537ad8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:01.206847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1107/2174] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:01.212511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cd681-e68c-4103-b100-5908e077de33 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:01.227208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cd681-e68c-4103-b100-5908e077de33 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:01.234155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f87601-4b56-44ed-9081-b4cf8ae9caa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:01.234836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f87601-4b56-44ed-9081-b4cf8ae9caa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:01.235109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f87601-4b56-44ed-9081-b4cf8ae9caa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:01.235448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f87601-4b56-44ed-9081-b4cf8ae9caa8 admin admin] PATH_INFO: `/v3/projects/44a1f7ec68db437eba7ced371b090141/users/0d85d45eac7347f0aa33c98f554e4dfd/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:01.253457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f87601-4b56-44ed-9081-b4cf8ae9caa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a1f7ec68db437eba7ced371b090141, user_id=0d85d45eac7347f0aa33c98f554e4dfd, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:01.256237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f87601-4b56-44ed-9081-b4cf8ae9caa8 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:01.361995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1068/2175] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:01 2022] PUT /identity/v3/projects/44a1f7ec68db437eba7ced371b090141/users/0d85d45eac7347f0aa33c98f554e4dfd/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:01.366543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd63d5f-929b-47e4-9a4b-e4416f537ad8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:01.377749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd63d5f-929b-47e4-9a4b-e4416f537ad8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:01.453852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f8261-5e61-45ed-9b8a-5ce60442d34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:01.453852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f8261-5e61-45ed-9b8a-5ce60442d34c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:01.453852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f8261-5e61-45ed-9b8a-5ce60442d34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:01.454166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f8261-5e61-45ed-9b8a-5ce60442d34c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:01.454643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f8261-5e61-45ed-9b8a-5ce60442d34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:01.456468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f8261-5e61-45ed-9b8a-5ce60442d34c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:01.471177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1108/2176] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:01.474250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f87601-4b56-44ed-9081-b4cf8ae9caa8 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:01.486270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f87601-4b56-44ed-9081-b4cf8ae9caa8 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:01.491295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae61a5-b852-4816-b20a-a648baf4d336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:01.514247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae61a5-b852-4816-b20a-a648baf4d336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:01.514247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae61a5-b852-4816-b20a-a648baf4d336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:01.514247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae61a5-b852-4816-b20a-a648baf4d336 admin admin] PATH_INFO: `/v3/projects/44a1f7ec68db437eba7ced371b090141/users/0d85d45eac7347f0aa33c98f554e4dfd/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:01.514247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae61a5-b852-4816-b20a-a648baf4d336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a1f7ec68db437eba7ced371b090141, user_id=0d85d45eac7347f0aa33c98f554e4dfd, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:01.514247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae61a5-b852-4816-b20a-a648baf4d336 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:01.514720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1069/2177] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:01 2022] PUT /identity/v3/projects/44a1f7ec68db437eba7ced371b090141/users/0d85d45eac7347f0aa33c98f554e4dfd/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:01.525432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33c83a-4a1a-438f-a42c-e0ee7f296996 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:01.525722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33c83a-4a1a-438f-a42c-e0ee7f296996 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:01.525999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33c83a-4a1a-438f-a42c-e0ee7f296996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:01.555832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ee33c83a-4a1a-438f-a42c-e0ee7f296996 None None] MFA Rules not processed for user `0d85d45eac7347f0aa33c98f554e4dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:01.557069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ee33c83a-4a1a-438f-a42c-e0ee7f296996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:01.629161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1109/2178] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:01.641782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae61a5-b852-4816-b20a-a648baf4d336 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:01.643528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-10ae61a5-b852-4816-b20a-a648baf4d336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:01.691302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:01.705452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae61a5-b852-4816-b20a-a648baf4d336 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:01.710091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985c5b0-096b-40cb-bc9b-7c96c9a14fd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:01.710881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985c5b0-096b-40cb-bc9b-7c96c9a14fd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:01.718160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985c5b0-096b-40cb-bc9b-7c96c9a14fd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:01.718160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985c5b0-096b-40cb-bc9b-7c96c9a14fd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:01.720910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985c5b0-096b-40cb-bc9b-7c96c9a14fd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:01.722361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985c5b0-096b-40cb-bc9b-7c96c9a14fd7 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:01.732842 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1070/2179] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:03.213581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33c83a-4a1a-438f-a42c-e0ee7f296996 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:03.224339 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33c83a-4a1a-438f-a42c-e0ee7f296996 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:03.229571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c763635-a6c9-4ee1-a872-fbb6bea510bf tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d85d45eac7347f0aa33c98f554e4dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44a1f7ec68db437eba7ced371b090141', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:03.230227 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c763635-a6c9-4ee1-a872-fbb6bea510bf tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:03.230510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c763635-a6c9-4ee1-a872-fbb6bea510bf tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:03.230849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c763635-a6c9-4ee1-a872-fbb6bea510bf tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:03.231470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c763635-a6c9-4ee1-a872-fbb6bea510bf tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:03.231922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c763635-a6c9-4ee1-a872-fbb6bea510bf tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:03.233705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c763635-a6c9-4ee1-a872-fbb6bea510bf tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:03.248343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1110/2180] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:36:03.253604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985c5b0-096b-40cb-bc9b-7c96c9a14fd7 service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:03.263816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985c5b0-096b-40cb-bc9b-7c96c9a14fd7 service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:03.268771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6be4-6ec2-43cd-a7d6-43b730b580a7 tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d85d45eac7347f0aa33c98f554e4dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44a1f7ec68db437eba7ced371b090141', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:03.269414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6be4-6ec2-43cd-a7d6-43b730b580a7 tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:03.269700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6be4-6ec2-43cd-a7d6-43b730b580a7 tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:03.269970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c6be4-6ec2-43cd-a7d6-43b730b580a7 tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:03.270599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c6be4-6ec2-43cd-a7d6-43b730b580a7 tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:03.273980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c6be4-6ec2-43cd-a7d6-43b730b580a7 tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:03.505798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1071/2181] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:03 2022] POST /identity/v3/projects => generated 354 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:03.510401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c763635-a6c9-4ee1-a872-fbb6bea510bf tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:03.520834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c763635-a6c9-4ee1-a872-fbb6bea510bf tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:03.545718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:03.594869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d85d45eac7347f0aa33c98f554e4dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44a1f7ec68db437eba7ced371b090141', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:03.595066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:03.595344 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:03.595615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:03.596296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:03.596582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:03.598458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:03.613290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1111/2182] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:36:03.679493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6be4-6ec2-43cd-a7d6-43b730b580a7 tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:03.692074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c6be4-6ec2-43cd-a7d6-43b730b580a7 tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:03.698326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d85d45eac7347f0aa33c98f554e4dfd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44a1f7ec68db437eba7ced371b090141', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:03.699006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:03.713641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:03.713641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] PATH_INFO: `/v3/projects/a089a71a095b40209d183de5b597a9f6` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:03.713641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a089a71a095b40209d183de5b597a9f6)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:03.713641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:04.023871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a089a71a095b40209d183de5b597a9f6', 'request_id': 'req-8f0fb180-6d7c-402d-a7be-1bd1df95278c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:04.030159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a089a71a095b40209d183de5b597a9f6', 'request_id': 'req-8f0fb180-6d7c-402d-a7be-1bd1df95278c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:04.195099 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a089a71a095b40209d183de5b597a9f6', 'request_id': 'req-8f0fb180-6d7c-402d-a7be-1bd1df95278c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:04.210809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] The token cache is being invalidate because project a089a71a095b40209d183de5b597a9f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:04.211057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:04.211734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8f0fb180-6d7c-402d-a7be-1bd1df95278c tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:04.213743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1072/2183] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:03 2022] DELETE /identity/v3/projects/a089a71a095b40209d183de5b597a9f6 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:05.690775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:05.693313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:05.776817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29db50-98e3-450b-b0ef-de83e77e3d9e tempest-VolumeQuotaClassesTest-174317419 tempest-VolumeQuotaClassesTest-174317419-project-admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:05.838117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:05.838859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:05.839128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:05.839398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] PATH_INFO: `/v3/users/337b7cd9c6fa47d3b176ea6389764f7b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:05.840556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=337b7cd9c6fa47d3b176ea6389764f7b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:05.842493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:06.346293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '337b7cd9c6fa47d3b176ea6389764f7b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.651042 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '337b7cd9c6fa47d3b176ea6389764f7b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.663706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:06.665935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '337b7cd9c6fa47d3b176ea6389764f7b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.666715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '337b7cd9c6fa47d3b176ea6389764f7b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.674655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '337b7cd9c6fa47d3b176ea6389764f7b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:06.681706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1112/2184] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:05 2022] DELETE /identity/v3/users/337b7cd9c6fa47d3b176ea6389764f7b => generated 0 bytes in 992 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:06.686273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c83b7-3b5b-47d3-8b2a-d1ca9a882891 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:06.686762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c83b7-3b5b-47d3-8b2a-d1ca9a882891 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:06.686871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c83b7-3b5b-47d3-8b2a-d1ca9a882891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:06.705553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-425c83b7-3b5b-47d3-8b2a-d1ca9a882891 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:06.706756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-425c83b7-3b5b-47d3-8b2a-d1ca9a882891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:06.770862 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1073/2185] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:06.779077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:06.786003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:06.844597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcad6ef-4fe4-42ce-8f06-1a4266c24ab9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:06.848393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622245b-701b-4b29-b8d6-a91161399dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:06.854530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622245b-701b-4b29-b8d6-a91161399dbd service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:06.854530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622245b-701b-4b29-b8d6-a91161399dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:06.854530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622245b-701b-4b29-b8d6-a91161399dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:06.859829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622245b-701b-4b29-b8d6-a91161399dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:06.861306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622245b-701b-4b29-b8d6-a91161399dbd service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:06.872271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1113/2186] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:07.727162 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c83b7-3b5b-47d3-8b2a-d1ca9a882891 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:07.728609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-425c83b7-3b5b-47d3-8b2a-d1ca9a882891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:07.743082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c83b7-3b5b-47d3-8b2a-d1ca9a882891 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:07.748652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:07.749421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:07.749717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:07.750008 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] PATH_INFO: `/v3/projects/8f59f8adebfc4ad8aea318ae32ce930e` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:07.751360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f59f8adebfc4ad8aea318ae32ce930e)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:07.753658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:07.916775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:07.930045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f59f8adebfc4ad8aea318ae32ce930e', 'request_id': 'req-206ee715-42c4-428f-80e9-1676da5a0372'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:07.935624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f59f8adebfc4ad8aea318ae32ce930e', 'request_id': 'req-206ee715-42c4-428f-80e9-1676da5a0372'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.097068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f59f8adebfc4ad8aea318ae32ce930e', 'request_id': 'req-206ee715-42c4-428f-80e9-1676da5a0372'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.110022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] The token cache is being invalidate because project 8f59f8adebfc4ad8aea318ae32ce930e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:08.110309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.110958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.112594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1074/2187] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:07 2022] DELETE /identity/v3/projects/8f59f8adebfc4ad8aea318ae32ce930e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:08.117527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622245b-701b-4b29-b8d6-a91161399dbd service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:08.127534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622245b-701b-4b29-b8d6-a91161399dbd service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:08.184843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:08.185387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:08.185661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:08.185987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] PATH_INFO: `/v3/users/0d85d45eac7347f0aa33c98f554e4dfd` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:08.188440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d85d45eac7347f0aa33c98f554e4dfd)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:08.189255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:08.638826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d85d45eac7347f0aa33c98f554e4dfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.856561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d85d45eac7347f0aa33c98f554e4dfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.874968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d85d45eac7347f0aa33c98f554e4dfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.875937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d85d45eac7347f0aa33c98f554e4dfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.884621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-5991790b-8537-4610-94fd-8f10be794d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d85d45eac7347f0aa33c98f554e4dfd'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:08.899276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1114/2188] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:08 2022] DELETE /identity/v3/users/0d85d45eac7347f0aa33c98f554e4dfd => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:09.527106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:09.528545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:09.543729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ee715-42c4-428f-80e9-1676da5a0372 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:09.603252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:09.603929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:09.604220 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:09.604491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] PATH_INFO: `/v3/projects/44a1f7ec68db437eba7ced371b090141` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:09.605702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a1f7ec68db437eba7ced371b090141)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:09.607646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:09.796628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a1f7ec68db437eba7ced371b090141', 'request_id': 'req-98324209-3884-42a4-98ea-a713fc63903a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:09.803296 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a1f7ec68db437eba7ced371b090141', 'request_id': 'req-98324209-3884-42a4-98ea-a713fc63903a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:09.977148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a1f7ec68db437eba7ced371b090141', 'request_id': 'req-98324209-3884-42a4-98ea-a713fc63903a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:09.991309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] The token cache is being invalidate because project 44a1f7ec68db437eba7ced371b090141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:09.991604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:09.992391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-98324209-3884-42a4-98ea-a713fc63903a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:09.994282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1075/2189] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:09 2022] DELETE /identity/v3/projects/44a1f7ec68db437eba7ced371b090141 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:10.000058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd328-9478-4e06-b489-3a168c23e1d8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.000531 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd328-9478-4e06-b489-3a168c23e1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.000679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd328-9478-4e06-b489-3a168c23e1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.015837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:10.022660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-624cd328-9478-4e06-b489-3a168c23e1d8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:10.024094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-624cd328-9478-4e06-b489-3a168c23e1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:10.094086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1115/2190] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:10.102520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6232d-f971-4823-8541-7cb64ab247b4 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.102803 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6232d-f971-4823-8541-7cb64ab247b4 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.103072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6232d-f971-4823-8541-7cb64ab247b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.123516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-f9d6232d-f971-4823-8541-7cb64ab247b4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:10.124727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f9d6232d-f971-4823-8541-7cb64ab247b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:10.133342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1076/2191] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:10.137815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd328-9478-4e06-b489-3a168c23e1d8 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:10.148722 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd328-9478-4e06-b489-3a168c23e1d8 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:10.153815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97577f38-4c72-4e54-8967-0e12677bcdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:10.154431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97577f38-4c72-4e54-8967-0e12677bcdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.154713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97577f38-4c72-4e54-8967-0e12677bcdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.154981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97577f38-4c72-4e54-8967-0e12677bcdd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.155661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97577f38-4c72-4e54-8967-0e12677bcdd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:10.155967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97577f38-4c72-4e54-8967-0e12677bcdd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:10.157769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97577f38-4c72-4e54-8967-0e12677bcdd9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:10.172491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1116/2192] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:36:10.178366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6232d-f971-4823-8541-7cb64ab247b4 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:10.187956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6232d-f971-4823-8541-7cb64ab247b4 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:10.192688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a2f6a-ab10-4018-ab08-8fff4701130a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:10.193356 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a2f6a-ab10-4018-ab08-8fff4701130a admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.193636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a2f6a-ab10-4018-ab08-8fff4701130a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.193910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a2f6a-ab10-4018-ab08-8fff4701130a admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.194705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a2f6a-ab10-4018-ab08-8fff4701130a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:10.196680 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a2f6a-ab10-4018-ab08-8fff4701130a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:10.407093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1077/2193] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:10 2022] POST /identity/v3/projects => generated 381 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:10.411517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97577f38-4c72-4e54-8967-0e12677bcdd9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:10.424341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97577f38-4c72-4e54-8967-0e12677bcdd9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:10.495860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1914d-870f-4870-8b6b-b2f51562cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:10.496527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1914d-870f-4870-8b6b-b2f51562cd63 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.496861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1914d-870f-4870-8b6b-b2f51562cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.497165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1914d-870f-4870-8b6b-b2f51562cd63 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.497847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1914d-870f-4870-8b6b-b2f51562cd63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:10.500024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1914d-870f-4870-8b6b-b2f51562cd63 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:10.821477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1117/2194] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:10 2022] POST /identity/v3/users => generated 291 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:10.821682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a2f6a-ab10-4018-ab08-8fff4701130a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:10.821682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a2f6a-ab10-4018-ab08-8fff4701130a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:10.821682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b548d1-ac61-4fbf-8ce6-f9b3f155ccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:10.821682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b548d1-ac61-4fbf-8ce6-f9b3f155ccb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.822421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b548d1-ac61-4fbf-8ce6-f9b3f155ccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.822421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b548d1-ac61-4fbf-8ce6-f9b3f155ccb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.822421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b548d1-ac61-4fbf-8ce6-f9b3f155ccb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:10.822421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b548d1-ac61-4fbf-8ce6-f9b3f155ccb7 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:10.822421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1078/2195] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:10.822822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1914d-870f-4870-8b6b-b2f51562cd63 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:10.822822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1914d-870f-4870-8b6b-b2f51562cd63 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:10.827449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb2eaa-970b-421d-9773-a3e74ed31a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:10.828057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb2eaa-970b-421d-9773-a3e74ed31a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.828337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb2eaa-970b-421d-9773-a3e74ed31a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.828602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb2eaa-970b-421d-9773-a3e74ed31a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.829282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb2eaa-970b-421d-9773-a3e74ed31a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:10.830891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb2eaa-970b-421d-9773-a3e74ed31a0d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:10.844299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1118/2196] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:10.849122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b548d1-ac61-4fbf-8ce6-f9b3f155ccb7 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:10.862358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b548d1-ac61-4fbf-8ce6-f9b3f155ccb7 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:10.870532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251d048-4d18-4967-aab5-efca3ce28de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:10.871621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251d048-4d18-4967-aab5-efca3ce28de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:10.872213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251d048-4d18-4967-aab5-efca3ce28de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:10.872616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251d048-4d18-4967-aab5-efca3ce28de6 admin admin] PATH_INFO: `/v3/projects/375f98eb8b504c8b8b17e58cdb9639de/users/1bf2b510d94f422ea4536397bb9dba52/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:10.888764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251d048-4d18-4967-aab5-efca3ce28de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375f98eb8b504c8b8b17e58cdb9639de, user_id=1bf2b510d94f422ea4536397bb9dba52, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:10.892593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251d048-4d18-4967-aab5-efca3ce28de6 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:11.133209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1079/2197] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:10 2022] PUT /identity/v3/projects/375f98eb8b504c8b8b17e58cdb9639de/users/1bf2b510d94f422ea4536397bb9dba52/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:11.144375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c0706-8901-4779-acd5-2057157d6a3f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:11.144786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c0706-8901-4779-acd5-2057157d6a3f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:11.145049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c0706-8901-4779-acd5-2057157d6a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:11.174852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-fc6c0706-8901-4779-acd5-2057157d6a3f None None] MFA Rules not processed for user `1bf2b510d94f422ea4536397bb9dba52`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:11.176069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fc6c0706-8901-4779-acd5-2057157d6a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:11.247242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1119/2198] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:11.251879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251d048-4d18-4967-aab5-efca3ce28de6 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:11.263748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:11.320093 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251d048-4d18-4967-aab5-efca3ce28de6 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:11.325039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d561d97-ebbb-4ff6-884d-470b95a5bb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:11.325727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d561d97-ebbb-4ff6-884d-470b95a5bb8c admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:11.325997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d561d97-ebbb-4ff6-884d-470b95a5bb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:11.326260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d561d97-ebbb-4ff6-884d-470b95a5bb8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:11.326912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d561d97-ebbb-4ff6-884d-470b95a5bb8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:11.328902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d561d97-ebbb-4ff6-884d-470b95a5bb8c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:11.595197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1080/2199] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:11 2022] POST /identity/v3/projects => generated 383 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:11.599070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c0706-8901-4779-acd5-2057157d6a3f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:11.610694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c0706-8901-4779-acd5-2057157d6a3f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:11.681697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138128f3-7727-4348-b66b-c629e4d8a7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:11.682465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138128f3-7727-4348-b66b-c629e4d8a7f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:11.682892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138128f3-7727-4348-b66b-c629e4d8a7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:11.683358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138128f3-7727-4348-b66b-c629e4d8a7f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:11.684144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138128f3-7727-4348-b66b-c629e4d8a7f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:11.686621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138128f3-7727-4348-b66b-c629e4d8a7f4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:11.887243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1120/2200] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:11 2022] POST /identity/v3/users => generated 298 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:11.891481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d561d97-ebbb-4ff6-884d-470b95a5bb8c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:11.903151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d561d97-ebbb-4ff6-884d-470b95a5bb8c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:11.908051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141572d-3bdd-4c56-8972-abd81d9d3814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:11.908648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141572d-3bdd-4c56-8972-abd81d9d3814 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:11.908966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141572d-3bdd-4c56-8972-abd81d9d3814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:11.909240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141572d-3bdd-4c56-8972-abd81d9d3814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:11.909872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141572d-3bdd-4c56-8972-abd81d9d3814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:11.911452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141572d-3bdd-4c56-8972-abd81d9d3814 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:11.922241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1081/2201] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:11.926592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138128f3-7727-4348-b66b-c629e4d8a7f4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:11.937595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138128f3-7727-4348-b66b-c629e4d8a7f4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:11.942221 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c283de2-ae2b-49b3-8da9-dc1eb35b294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:11.942864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283de2-ae2b-49b3-8da9-dc1eb35b294b admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:11.943838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283de2-ae2b-49b3-8da9-dc1eb35b294b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:11.944220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283de2-ae2b-49b3-8da9-dc1eb35b294b admin admin] PATH_INFO: `/v3/projects/f1b2e4b8c6064327a1e07c4cf7a3aa5b/users/fe1ad8e1874540e0b6a7547348ec6deb/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:11.964903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c283de2-ae2b-49b3-8da9-dc1eb35b294b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b2e4b8c6064327a1e07c4cf7a3aa5b, user_id=fe1ad8e1874540e0b6a7547348ec6deb, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:11.968384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c283de2-ae2b-49b3-8da9-dc1eb35b294b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:12.247465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1121/2202] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:11 2022] PUT /identity/v3/projects/f1b2e4b8c6064327a1e07c4cf7a3aa5b/users/fe1ad8e1874540e0b6a7547348ec6deb/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:12.251574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141572d-3bdd-4c56-8972-abd81d9d3814 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:12.262301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141572d-3bdd-4c56-8972-abd81d9d3814 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:12.322345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3736a6a-ea67-4229-8f1d-b3756efa71a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:12.322967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736a6a-ea67-4229-8f1d-b3756efa71a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:12.323246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736a6a-ea67-4229-8f1d-b3756efa71a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:12.323519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736a6a-ea67-4229-8f1d-b3756efa71a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:12.324165 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3736a6a-ea67-4229-8f1d-b3756efa71a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:12.325802 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3736a6a-ea67-4229-8f1d-b3756efa71a3 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:12.337003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1082/2203] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:12.341802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c283de2-ae2b-49b3-8da9-dc1eb35b294b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:12.353848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c283de2-ae2b-49b3-8da9-dc1eb35b294b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:12.358476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cc0be-8abf-4e26-b58c-f8a37fc4fa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:12.359606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cc0be-8abf-4e26-b58c-f8a37fc4fa42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:12.359916 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cc0be-8abf-4e26-b58c-f8a37fc4fa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:12.360224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cc0be-8abf-4e26-b58c-f8a37fc4fa42 admin admin] PATH_INFO: `/v3/projects/f1b2e4b8c6064327a1e07c4cf7a3aa5b/users/fe1ad8e1874540e0b6a7547348ec6deb/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:12.362402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cc0be-8abf-4e26-b58c-f8a37fc4fa42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b2e4b8c6064327a1e07c4cf7a3aa5b, user_id=fe1ad8e1874540e0b6a7547348ec6deb, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:12.366175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cc0be-8abf-4e26-b58c-f8a37fc4fa42 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:12.372864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:12.381507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1122/2204] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:12 2022] PUT /identity/v3/projects/f1b2e4b8c6064327a1e07c4cf7a3aa5b/users/fe1ad8e1874540e0b6a7547348ec6deb/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:12.392027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12d3b9-8ca5-48a4-8236-fdcf382c0a3c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:12.392306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12d3b9-8ca5-48a4-8236-fdcf382c0a3c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:12.392575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12d3b9-8ca5-48a4-8236-fdcf382c0a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:12.425808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6e12d3b9-8ca5-48a4-8236-fdcf382c0a3c None None] MFA Rules not processed for user `fe1ad8e1874540e0b6a7547348ec6deb`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:12.425808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6e12d3b9-8ca5-48a4-8236-fdcf382c0a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:12.493652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1083/2205] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:12.506914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cc0be-8abf-4e26-b58c-f8a37fc4fa42 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:12.516133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c10cc0be-8abf-4e26-b58c-f8a37fc4fa42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:12.570633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cc0be-8abf-4e26-b58c-f8a37fc4fa42 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:12.574238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d485a-8d24-4c3d-af47-f8330cc34cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:12.574876 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d485a-8d24-4c3d-af47-f8330cc34cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:12.575151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d485a-8d24-4c3d-af47-f8330cc34cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:12.575423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d485a-8d24-4c3d-af47-f8330cc34cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:12.585486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d485a-8d24-4c3d-af47-f8330cc34cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:12.586975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d485a-8d24-4c3d-af47-f8330cc34cc5 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:12.597647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1123/2206] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:12.939302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12d3b9-8ca5-48a4-8236-fdcf382c0a3c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:12.940738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6e12d3b9-8ca5-48a4-8236-fdcf382c0a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:12.953121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12d3b9-8ca5-48a4-8236-fdcf382c0a3c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:12.956502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be09f8-966f-4ad4-9444-5318c27bba4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:12.957125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be09f8-966f-4ad4-9444-5318c27bba4e service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:12.957414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be09f8-966f-4ad4-9444-5318c27bba4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:12.957666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be09f8-966f-4ad4-9444-5318c27bba4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:13.016290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be09f8-966f-4ad4-9444-5318c27bba4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:13.018149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6be09f8-966f-4ad4-9444-5318c27bba4e service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:13.027363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1084/2207] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:13.136405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:14.093847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d485a-8d24-4c3d-af47-f8330cc34cc5 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:14.165995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:14.172352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d485a-8d24-4c3d-af47-f8330cc34cc5 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:14.177359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:14.179298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:14.179298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:14.179298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] PATH_INFO: `/v3/users/1bf2b510d94f422ea4536397bb9dba52` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:14.180880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf2b510d94f422ea4536397bb9dba52)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:14.182821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:14.493836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf2b510d94f422ea4536397bb9dba52'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.627365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf2b510d94f422ea4536397bb9dba52'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.642589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf2b510d94f422ea4536397bb9dba52'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.643263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf2b510d94f422ea4536397bb9dba52'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.651338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf2b510d94f422ea4536397bb9dba52'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:14.657638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1124/2208] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:14 2022] DELETE /identity/v3/users/1bf2b510d94f422ea4536397bb9dba52 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:14.663442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca51bb-04e6-44bb-83bc-5557d8e83fc3 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:14.663975 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca51bb-04e6-44bb-83bc-5557d8e83fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:14.663975 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca51bb-04e6-44bb-83bc-5557d8e83fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:14.685513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e6ca51bb-04e6-44bb-83bc-5557d8e83fc3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:14.687115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e6ca51bb-04e6-44bb-83bc-5557d8e83fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:14.755181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1085/2209] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:14.765021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:14.766649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:14.831366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b17ec-9569-4255-939c-384b3d7e91c8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:14.836080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ca90c-ecde-4b22-89ea-8a9c2833cc90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:14.836693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ca90c-ecde-4b22-89ea-8a9c2833cc90 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:14.837003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ca90c-ecde-4b22-89ea-8a9c2833cc90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:14.837277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ca90c-ecde-4b22-89ea-8a9c2833cc90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:14.847415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ca90c-ecde-4b22-89ea-8a9c2833cc90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:14.848906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ca90c-ecde-4b22-89ea-8a9c2833cc90 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:14.860027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1125/2210] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:15.424084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca51bb-04e6-44bb-83bc-5557d8e83fc3 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:15.426464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e6ca51bb-04e6-44bb-83bc-5557d8e83fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:15.440043 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca51bb-04e6-44bb-83bc-5557d8e83fc3 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:15.446139 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:15.446789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:15.447065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:15.447333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] PATH_INFO: `/v3/projects/375f98eb8b504c8b8b17e58cdb9639de` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:15.449404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375f98eb8b504c8b8b17e58cdb9639de)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:15.451202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:15.630355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375f98eb8b504c8b8b17e58cdb9639de', 'request_id': 'req-aa29f7da-2a03-4f29-a053-446b92c0a369'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:15.636763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375f98eb8b504c8b8b17e58cdb9639de', 'request_id': 'req-aa29f7da-2a03-4f29-a053-446b92c0a369'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:15.776457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375f98eb8b504c8b8b17e58cdb9639de', 'request_id': 'req-aa29f7da-2a03-4f29-a053-446b92c0a369'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:15.790750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] The token cache is being invalidate because project 375f98eb8b504c8b8b17e58cdb9639de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:15.791194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:15.791948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:15.793743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1086/2211] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:15 2022] DELETE /identity/v3/projects/375f98eb8b504c8b8b17e58cdb9639de => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:15.799091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ca90c-ecde-4b22-89ea-8a9c2833cc90 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:15.810361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ca90c-ecde-4b22-89ea-8a9c2833cc90 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:15.875713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:15.876343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:15.876618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:15.876929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] PATH_INFO: `/v3/users/fe1ad8e1874540e0b6a7547348ec6deb` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:15.878059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe1ad8e1874540e0b6a7547348ec6deb)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:15.879966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:15.903483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:16.201796 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe1ad8e1874540e0b6a7547348ec6deb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:16.307666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe1ad8e1874540e0b6a7547348ec6deb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:16.321925 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe1ad8e1874540e0b6a7547348ec6deb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:16.322664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe1ad8e1874540e0b6a7547348ec6deb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:16.329930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-64bcdc1f-c4b6-4141-9ef3-da4d6f7a1c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe1ad8e1874540e0b6a7547348ec6deb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:16.336993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1126/2212] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:15 2022] DELETE /identity/v3/users/fe1ad8e1874540e0b6a7547348ec6deb => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:16.926618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:16.928246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:16.953603 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29f7da-2a03-4f29-a053-446b92c0a369 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:16.965155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:17.029111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:17.029867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:17.030094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:17.030369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] PATH_INFO: `/v3/projects/f1b2e4b8c6064327a1e07c4cf7a3aa5b` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:17.031516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b2e4b8c6064327a1e07c4cf7a3aa5b)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:17.033635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:17.220463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b2e4b8c6064327a1e07c4cf7a3aa5b', 'request_id': 'req-8bbde86b-b919-431a-863f-e3dbda3d7aeb'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:17.227179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b2e4b8c6064327a1e07c4cf7a3aa5b', 'request_id': 'req-8bbde86b-b919-431a-863f-e3dbda3d7aeb'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:17.394201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b2e4b8c6064327a1e07c4cf7a3aa5b', 'request_id': 'req-8bbde86b-b919-431a-863f-e3dbda3d7aeb'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:17.408933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] The token cache is being invalidate because project f1b2e4b8c6064327a1e07c4cf7a3aa5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:17.409352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:17.410203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bbde86b-b919-431a-863f-e3dbda3d7aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:17.411805 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1087/2213] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:16 2022] DELETE /identity/v3/projects/f1b2e4b8c6064327a1e07c4cf7a3aa5b => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:17.418275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb02d5-72c3-4ded-9322-5cb16a660c2d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:17.418982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb02d5-72c3-4ded-9322-5cb16a660c2d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:17.418982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb02d5-72c3-4ded-9322-5cb16a660c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:17.440333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-7abb02d5-72c3-4ded-9322-5cb16a660c2d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:17.441592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-7abb02d5-72c3-4ded-9322-5cb16a660c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:17.504563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1127/2214] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:17.513566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d9c9a-07b5-41ed-879a-9ab2bb8cf552 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:17.513855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d9c9a-07b5-41ed-879a-9ab2bb8cf552 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:17.514136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d9c9a-07b5-41ed-879a-9ab2bb8cf552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:17.534253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-016d9c9a-07b5-41ed-879a-9ab2bb8cf552 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:17.535477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-016d9c9a-07b5-41ed-879a-9ab2bb8cf552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:17.543881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1088/2215] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:17.548511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb02d5-72c3-4ded-9322-5cb16a660c2d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:17.559251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb02d5-72c3-4ded-9322-5cb16a660c2d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:17.563841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d25fd5-f192-47d2-a134-70e9fef7bff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:17.564433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d25fd5-f192-47d2-a134-70e9fef7bff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:17.564709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d25fd5-f192-47d2-a134-70e9fef7bff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:17.565018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d25fd5-f192-47d2-a134-70e9fef7bff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:17.565662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d25fd5-f192-47d2-a134-70e9fef7bff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:17.565961 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d25fd5-f192-47d2-a134-70e9fef7bff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:17.567545 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d25fd5-f192-47d2-a134-70e9fef7bff0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:17.581102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1128/2216] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:36:17.585785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d9c9a-07b5-41ed-879a-9ab2bb8cf552 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:17.595143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016d9c9a-07b5-41ed-879a-9ab2bb8cf552 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:17.600593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba09583-4ed7-4b75-be0f-186cdcc3d6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:17.601226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba09583-4ed7-4b75-be0f-186cdcc3d6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:17.601505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba09583-4ed7-4b75-be0f-186cdcc3d6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:17.601794 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba09583-4ed7-4b75-be0f-186cdcc3d6d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:17.602427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba09583-4ed7-4b75-be0f-186cdcc3d6d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:17.604392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba09583-4ed7-4b75-be0f-186cdcc3d6d7 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:17.812317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1089/2217] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:17 2022] POST /identity/v3/projects => generated 375 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:17.816255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d25fd5-f192-47d2-a134-70e9fef7bff0 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:17.827675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d25fd5-f192-47d2-a134-70e9fef7bff0 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:17.889512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4515c-f03e-4dfb-8ceb-050ecceb8aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:17.890130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4515c-f03e-4dfb-8ceb-050ecceb8aed admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:17.890428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4515c-f03e-4dfb-8ceb-050ecceb8aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:17.890698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4515c-f03e-4dfb-8ceb-050ecceb8aed admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:17.891366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4515c-f03e-4dfb-8ceb-050ecceb8aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:17.893786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4515c-f03e-4dfb-8ceb-050ecceb8aed admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:17.996637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1129/2218] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:17 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:18.003096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba09583-4ed7-4b75-be0f-186cdcc3d6d7 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:18.016941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba09583-4ed7-4b75-be0f-186cdcc3d6d7 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:18.022478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641edb4-b01f-4db4-9106-d755e8c504c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:18.023156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641edb4-b01f-4db4-9106-d755e8c504c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:18.023437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641edb4-b01f-4db4-9106-d755e8c504c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:18.023711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641edb4-b01f-4db4-9106-d755e8c504c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:18.024362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641edb4-b01f-4db4-9106-d755e8c504c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:18.025987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641edb4-b01f-4db4-9106-d755e8c504c5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:18.037737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1090/2219] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:18.041485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4515c-f03e-4dfb-8ceb-050ecceb8aed admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:18.051863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4515c-f03e-4dfb-8ceb-050ecceb8aed admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:18.057252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9035f67-1d46-45f5-b207-28faaacc3942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:18.059893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9035f67-1d46-45f5-b207-28faaacc3942 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:18.059893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9035f67-1d46-45f5-b207-28faaacc3942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:18.059893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9035f67-1d46-45f5-b207-28faaacc3942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:18.059893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9035f67-1d46-45f5-b207-28faaacc3942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:18.060597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9035f67-1d46-45f5-b207-28faaacc3942 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:18.071951 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1130/2220] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:18.076299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641edb4-b01f-4db4-9106-d755e8c504c5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:18.087767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641edb4-b01f-4db4-9106-d755e8c504c5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:18.097525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b771c0-9ac0-4bbb-ab8a-be5060807ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:18.097525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b771c0-9ac0-4bbb-ab8a-be5060807ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:18.097525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b771c0-9ac0-4bbb-ab8a-be5060807ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:18.097525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b771c0-9ac0-4bbb-ab8a-be5060807ed5 admin admin] PATH_INFO: `/v3/projects/26d47fa8e7854ec2abb80e0fb593f483/users/7f7fcfe552ba47039135a49549fef7df/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:18.110243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b771c0-9ac0-4bbb-ab8a-be5060807ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d47fa8e7854ec2abb80e0fb593f483, user_id=7f7fcfe552ba47039135a49549fef7df, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:18.113021 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b771c0-9ac0-4bbb-ab8a-be5060807ed5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:18.337238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1091/2221] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:18 2022] PUT /identity/v3/projects/26d47fa8e7854ec2abb80e0fb593f483/users/7f7fcfe552ba47039135a49549fef7df/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:18.407203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04712a-1365-4f94-a900-719268823cae None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:18.407484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04712a-1365-4f94-a900-719268823cae None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:18.407759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04712a-1365-4f94-a900-719268823cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:18.417053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:18.440987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ed04712a-1365-4f94-a900-719268823cae None None] MFA Rules not processed for user `7f7fcfe552ba47039135a49549fef7df`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:18.443181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ed04712a-1365-4f94-a900-719268823cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:18.519692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1131/2222] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:18.531261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b771c0-9ac0-4bbb-ab8a-be5060807ed5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:18.533152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b9b771c0-9ac0-4bbb-ab8a-be5060807ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:18.579398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:18.597364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b771c0-9ac0-4bbb-ab8a-be5060807ed5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:18.602754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2109088-2505-44e0-b344-0b5795da1716 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:18.603430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2109088-2505-44e0-b344-0b5795da1716 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:18.603702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2109088-2505-44e0-b344-0b5795da1716 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:18.603967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2109088-2505-44e0-b344-0b5795da1716 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:18.617102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2109088-2505-44e0-b344-0b5795da1716 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:18.618593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2109088-2505-44e0-b344-0b5795da1716 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:18.628630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1092/2223] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:24.522354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04712a-1365-4f94-a900-719268823cae None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:24.599554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04712a-1365-4f94-a900-719268823cae None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:24.605151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:24.605787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:24.606059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:24.606323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] PATH_INFO: `/v3/users/7f7fcfe552ba47039135a49549fef7df` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:24.607644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7fcfe552ba47039135a49549fef7df)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:24.609745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:24.820031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2109088-2505-44e0-b344-0b5795da1716 service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:24.820031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f2109088-2505-44e0-b344-0b5795da1716 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:24.832897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2109088-2505-44e0-b344-0b5795da1716 service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:24.837814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:24.838438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:24.838751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:24.839071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] PATH_INFO: `/v3/users/46769294716e417dab410cb82d18c066` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:24.840446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46769294716e417dab410cb82d18c066)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:24.842626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:24.923017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7fcfe552ba47039135a49549fef7df'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.067724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7fcfe552ba47039135a49549fef7df'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.082181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7fcfe552ba47039135a49549fef7df'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.082873 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7fcfe552ba47039135a49549fef7df'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.090676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f8dae085-3af7-4fcc-9acc-cc7e8e0e3ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7fcfe552ba47039135a49549fef7df'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.096617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1132/2224] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:24 2022] DELETE /identity/v3/users/7f7fcfe552ba47039135a49549fef7df => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:25.103653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890b7c-b5ba-41e8-a596-5ed09d429179 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.112849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890b7c-b5ba-41e8-a596-5ed09d429179 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.112849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890b7c-b5ba-41e8-a596-5ed09d429179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.129712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8d890b7c-b5ba-41e8-a596-5ed09d429179 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:25.130963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8d890b7c-b5ba-41e8-a596-5ed09d429179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:25.140231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:25.165519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46769294716e417dab410cb82d18c066'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.166295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46769294716e417dab410cb82d18c066'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.166995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46769294716e417dab410cb82d18c066'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.174403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46769294716e417dab410cb82d18c066'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.187318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46769294716e417dab410cb82d18c066'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.200048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1133/2225] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:25.211548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d890b7c-b5ba-41e8-a596-5ed09d429179 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:25.213043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8d890b7c-b5ba-41e8-a596-5ed09d429179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:25.274859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d890b7c-b5ba-41e8-a596-5ed09d429179 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:25.278953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66616f-132a-4429-a06c-2fe440b8ea77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:25.279712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66616f-132a-4429-a06c-2fe440b8ea77 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.281658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66616f-132a-4429-a06c-2fe440b8ea77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.282319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66616f-132a-4429-a06c-2fe440b8ea77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.294302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66616f-132a-4429-a06c-2fe440b8ea77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:25.297033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66616f-132a-4429-a06c-2fe440b8ea77 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:25.312514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1134/2226] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:25.337677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1093/2227] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:24 2022] DELETE /identity/v3/users/46769294716e417dab410cb82d18c066 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:36:25.344678 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166efa37-3e0e-41a0-91c3-52b749bb2b7b None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.345203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166efa37-3e0e-41a0-91c3-52b749bb2b7b None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.345943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166efa37-3e0e-41a0-91c3-52b749bb2b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.369786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-166efa37-3e0e-41a0-91c3-52b749bb2b7b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:25.371305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-166efa37-3e0e-41a0-91c3-52b749bb2b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:25.446092 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1135/2228] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:25.456894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:25.522682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc283729-56e5-4f61-b372-31a2aef30b97 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:25.526950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d043f44-7746-4a8b-98e7-36db14865ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:25.527582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d043f44-7746-4a8b-98e7-36db14865ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.527867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d043f44-7746-4a8b-98e7-36db14865ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.528137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d043f44-7746-4a8b-98e7-36db14865ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.544126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d043f44-7746-4a8b-98e7-36db14865ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:25.545738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d043f44-7746-4a8b-98e7-36db14865ec3 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:25.558480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1094/2229] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:25.754830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166efa37-3e0e-41a0-91c3-52b749bb2b7b None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:25.759094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-166efa37-3e0e-41a0-91c3-52b749bb2b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:25.774575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166efa37-3e0e-41a0-91c3-52b749bb2b7b None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:25.780313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:25.781113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:25.781485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:25.781790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] PATH_INFO: `/v3/projects/26d47fa8e7854ec2abb80e0fb593f483` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:25.783904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d47fa8e7854ec2abb80e0fb593f483)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:25.788903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:25.909786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d47fa8e7854ec2abb80e0fb593f483', 'request_id': 'req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:25.912135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:25.919380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d47fa8e7854ec2abb80e0fb593f483', 'request_id': 'req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:26.005427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d47fa8e7854ec2abb80e0fb593f483', 'request_id': 'req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:26.012693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d043f44-7746-4a8b-98e7-36db14865ec3 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.014464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8d043f44-7746-4a8b-98e7-36db14865ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:26.026801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] The token cache is being invalidate because project 26d47fa8e7854ec2abb80e0fb593f483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:26.027265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:26.028065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-a65ce416-b8a2-4cb1-a0b7-ed2c786a1264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:26.030744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1136/2230] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:25 2022] DELETE /identity/v3/projects/26d47fa8e7854ec2abb80e0fb593f483 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:26.038612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d043f44-7746-4a8b-98e7-36db14865ec3 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.044142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dded3-8bdf-4d92-b148-0242f27833c0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.045010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dded3-8bdf-4d92-b148-0242f27833c0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.045508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dded3-8bdf-4d92-b148-0242f27833c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.074740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c36dded3-8bdf-4d92-b148-0242f27833c0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:26.076167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c36dded3-8bdf-4d92-b148-0242f27833c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:26.108557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:26.116671 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.117562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.117814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.118082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] PATH_INFO: `/v3/projects/191948ea02004177a488b2483cb1607b` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.119233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] RBAC: Authorizing `identity:delete_project(project_id=191948ea02004177a488b2483cb1607b)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.121441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.145277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1137/2231] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:26.153951 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a6984-54cd-41fa-9e96-a1e8bdb7b6b3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.154238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a6984-54cd-41fa-9e96-a1e8bdb7b6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.154512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a6984-54cd-41fa-9e96-a1e8bdb7b6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.174958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b63a6984-54cd-41fa-9e96-a1e8bdb7b6b3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:26.182159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b63a6984-54cd-41fa-9e96-a1e8bdb7b6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:26.184677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1138/2232] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:26.189082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a6984-54cd-41fa-9e96-a1e8bdb7b6b3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.200047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a6984-54cd-41fa-9e96-a1e8bdb7b6b3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.204953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de567760-4183-493f-b014-49dcb0fea2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.210897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de567760-4183-493f-b014-49dcb0fea2ae admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.210897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de567760-4183-493f-b014-49dcb0fea2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.210897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de567760-4183-493f-b014-49dcb0fea2ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.210897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de567760-4183-493f-b014-49dcb0fea2ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:26.210897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de567760-4183-493f-b014-49dcb0fea2ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.210897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de567760-4183-493f-b014-49dcb0fea2ae admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.222687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1139/2233] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:36:26.227268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de567760-4183-493f-b014-49dcb0fea2ae admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.237781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de567760-4183-493f-b014-49dcb0fea2ae admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.242827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c0f1c-1350-47fd-b39e-39f23f7af3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.243442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0f1c-1350-47fd-b39e-39f23f7af3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.243724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0f1c-1350-47fd-b39e-39f23f7af3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.243995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c0f1c-1350-47fd-b39e-39f23f7af3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.244655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c0f1c-1350-47fd-b39e-39f23f7af3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.246878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c0f1c-1350-47fd-b39e-39f23f7af3a9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.327569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '191948ea02004177a488b2483cb1607b', 'request_id': 'req-d3d1a69f-565a-48e4-9977-c7525e14a29a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:26.339256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '191948ea02004177a488b2483cb1607b', 'request_id': 'req-d3d1a69f-565a-48e4-9977-c7525e14a29a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:26.423236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1140/2234] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:26 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:26.427249 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c0f1c-1350-47fd-b39e-39f23f7af3a9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.439463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '191948ea02004177a488b2483cb1607b', 'request_id': 'req-d3d1a69f-565a-48e4-9977-c7525e14a29a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:26.458748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] The token cache is being invalidate because project 191948ea02004177a488b2483cb1607b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:26.459038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:26.459797 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d3d1a69f-565a-48e4-9977-c7525e14a29a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:26.461392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1095/2235] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:26 2022] DELETE /identity/v3/projects/191948ea02004177a488b2483cb1607b => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:26.466046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92f4b-80b2-46db-8de7-ca2510e3abfb None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.466329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92f4b-80b2-46db-8de7-ca2510e3abfb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.466604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92f4b-80b2-46db-8de7-ca2510e3abfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.487213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-21a92f4b-80b2-46db-8de7-ca2510e3abfb None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:26.488430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-21a92f4b-80b2-46db-8de7-ca2510e3abfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:26.497548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1096/2236] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:26.499893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c0f1c-1350-47fd-b39e-39f23f7af3a9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.504676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b028fb-1e3d-4953-ac6e-eef92f75c204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.505344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c673b22-4110-4efb-933a-15d4698b8403 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.505670 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b028fb-1e3d-4953-ac6e-eef92f75c204 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.505816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c673b22-4110-4efb-933a-15d4698b8403 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.505903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b028fb-1e3d-4953-ac6e-eef92f75c204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.506272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c673b22-4110-4efb-933a-15d4698b8403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.506447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b028fb-1e3d-4953-ac6e-eef92f75c204 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.507783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b028fb-1e3d-4953-ac6e-eef92f75c204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.509841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b028fb-1e3d-4953-ac6e-eef92f75c204 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.536047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9c673b22-4110-4efb-933a-15d4698b8403 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:26.537294 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9c673b22-4110-4efb-933a-15d4698b8403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:26.549108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1097/2237] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:36:26.555746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c673b22-4110-4efb-933a-15d4698b8403 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.565640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c673b22-4110-4efb-933a-15d4698b8403 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.571314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c121-d5af-4524-a7e2-124c62fbcae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.571919 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c121-d5af-4524-a7e2-124c62fbcae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.572194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c121-d5af-4524-a7e2-124c62fbcae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.572464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c121-d5af-4524-a7e2-124c62fbcae9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.573154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281c121-d5af-4524-a7e2-124c62fbcae9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:36:26.573449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281c121-d5af-4524-a7e2-124c62fbcae9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.575047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281c121-d5af-4524-a7e2-124c62fbcae9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.587614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1098/2238] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:36:26.593311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c121-d5af-4524-a7e2-124c62fbcae9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.604121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1141/2239] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:26 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:26.604338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c121-d5af-4524-a7e2-124c62fbcae9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.605252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b028fb-1e3d-4953-ac6e-eef92f75c204 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.607062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-22b028fb-1e3d-4953-ac6e-eef92f75c204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:26.608230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d0a90-ae0e-41a2-8ee3-d5aed5f2b3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.608839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d0a90-ae0e-41a2-8ee3-d5aed5f2b3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.616619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d0a90-ae0e-41a2-8ee3-d5aed5f2b3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.616619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d0a90-ae0e-41a2-8ee3-d5aed5f2b3b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.616619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d0a90-ae0e-41a2-8ee3-d5aed5f2b3b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.616619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d0a90-ae0e-41a2-8ee3-d5aed5f2b3b3 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.621771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b028fb-1e3d-4953-ac6e-eef92f75c204 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.626667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd96a2-82c6-49fa-b730-2d253b8451ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.627268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd96a2-82c6-49fa-b730-2d253b8451ab admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.627544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd96a2-82c6-49fa-b730-2d253b8451ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.627814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd96a2-82c6-49fa-b730-2d253b8451ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.628447 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd96a2-82c6-49fa-b730-2d253b8451ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.630303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd96a2-82c6-49fa-b730-2d253b8451ab admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.641500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1142/2240] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:36:26.645492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd96a2-82c6-49fa-b730-2d253b8451ab admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.656473 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd96a2-82c6-49fa-b730-2d253b8451ab admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.661159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb4a5e-f6b0-4e7e-81bd-80606fba72bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.661755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb4a5e-f6b0-4e7e-81bd-80606fba72bd admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.662036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb4a5e-f6b0-4e7e-81bd-80606fba72bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.662310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb4a5e-f6b0-4e7e-81bd-80606fba72bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.663608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb4a5e-f6b0-4e7e-81bd-80606fba72bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.665259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb4a5e-f6b0-4e7e-81bd-80606fba72bd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.676353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1143/2241] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:36:26.681462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb4a5e-f6b0-4e7e-81bd-80606fba72bd admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.691245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb4a5e-f6b0-4e7e-81bd-80606fba72bd admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.697264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59bc84-9668-4936-8a38-beddfc7636f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.697907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59bc84-9668-4936-8a38-beddfc7636f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.698186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59bc84-9668-4936-8a38-beddfc7636f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.698460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59bc84-9668-4936-8a38-beddfc7636f7 admin admin] PATH_INFO: `/v3/projects/c4ef3fc15a0147e39228729d2b5b2631/users/0b82b6f568bd41fa9d6b44facb766e0b/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.700725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:26.714656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59bc84-9668-4936-8a38-beddfc7636f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ef3fc15a0147e39228729d2b5b2631, user_id=0b82b6f568bd41fa9d6b44facb766e0b, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.717413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59bc84-9668-4936-8a38-beddfc7636f7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.750619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1099/2242] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:26 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:26.755559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d0a90-ae0e-41a2-8ee3-d5aed5f2b3b3 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.816148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1144/2243] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:26 2022] PUT /identity/v3/projects/c4ef3fc15a0147e39228729d2b5b2631/users/0b82b6f568bd41fa9d6b44facb766e0b/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:36:26.819670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d0a90-ae0e-41a2-8ee3-d5aed5f2b3b3 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:26.822393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b79c1-d094-4238-b25d-f53eb20d7502 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.822903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b79c1-d094-4238-b25d-f53eb20d7502 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.823218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b79c1-d094-4238-b25d-f53eb20d7502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.844572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a76b79c1-d094-4238-b25d-f53eb20d7502 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:26.845844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a76b79c1-d094-4238-b25d-f53eb20d7502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:26.871417 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:26.890595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac15d1-dbec-4443-998b-56e076774410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:26.890595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac15d1-dbec-4443-998b-56e076774410 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:26.890595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac15d1-dbec-4443-998b-56e076774410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:26.890595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac15d1-dbec-4443-998b-56e076774410 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:26.891204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac15d1-dbec-4443-998b-56e076774410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:26.893136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac15d1-dbec-4443-998b-56e076774410 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:26.926969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1145/2244] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:26.939104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b79c1-d094-4238-b25d-f53eb20d7502 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:26.940999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a76b79c1-d094-4238-b25d-f53eb20d7502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:26.994126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1100/2245] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:26 2022] POST /identity/v3/users => generated 287 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:26.998996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac15d1-dbec-4443-998b-56e076774410 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:27.007660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b79c1-d094-4238-b25d-f53eb20d7502 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:27.011271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac15d1-dbec-4443-998b-56e076774410 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:27.011998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac57f3d-06af-4bf5-a771-479437c2d18c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:27.012633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac57f3d-06af-4bf5-a771-479437c2d18c service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:27.012945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac57f3d-06af-4bf5-a771-479437c2d18c service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:27.013241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac57f3d-06af-4bf5-a771-479437c2d18c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:27.016551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0ca35-22c4-4ec9-981d-6fdefc106238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:27.017535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0ca35-22c4-4ec9-981d-6fdefc106238 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:27.017852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0ca35-22c4-4ec9-981d-6fdefc106238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:27.018126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0ca35-22c4-4ec9-981d-6fdefc106238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:27.018937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0ca35-22c4-4ec9-981d-6fdefc106238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:27.020562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0ca35-22c4-4ec9-981d-6fdefc106238 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:27.023520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac57f3d-06af-4bf5-a771-479437c2d18c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:27.024996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac57f3d-06af-4bf5-a771-479437c2d18c service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:27.034269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1101/2246] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:36:27.035603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1146/2247] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:36:27.038565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0ca35-22c4-4ec9-981d-6fdefc106238 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:27.049601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0ca35-22c4-4ec9-981d-6fdefc106238 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:27.057382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd241c-9a53-49b1-a536-8d83094e8fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:27.058143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd241c-9a53-49b1-a536-8d83094e8fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:27.058616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd241c-9a53-49b1-a536-8d83094e8fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:27.058988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd241c-9a53-49b1-a536-8d83094e8fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:27.059929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd241c-9a53-49b1-a536-8d83094e8fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:27.062414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd241c-9a53-49b1-a536-8d83094e8fb3 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:27.074750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1102/2248] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:36:27.081885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac57f3d-06af-4bf5-a771-479437c2d18c service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:27.092909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac57f3d-06af-4bf5-a771-479437c2d18c service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:27.097824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7241325-2cbf-4c42-a2e6-938c167d5cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:27.098497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7241325-2cbf-4c42-a2e6-938c167d5cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:27.098748 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7241325-2cbf-4c42-a2e6-938c167d5cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:27.099018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7241325-2cbf-4c42-a2e6-938c167d5cbb admin admin] PATH_INFO: `/v3/projects/5137e6ff4dea4dea9b31f988865d68de/users/c63276ee73414ea6a7fde01990893bfb/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:27.118249 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7241325-2cbf-4c42-a2e6-938c167d5cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5137e6ff4dea4dea9b31f988865d68de, user_id=c63276ee73414ea6a7fde01990893bfb, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:27.121228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7241325-2cbf-4c42-a2e6-938c167d5cbb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:27.200667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1147/2249] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:27 2022] PUT /identity/v3/projects/5137e6ff4dea4dea9b31f988865d68de/users/c63276ee73414ea6a7fde01990893bfb/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:27.212560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a14f5-e54b-476e-adda-c37c0ded0c81 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:27.212870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a14f5-e54b-476e-adda-c37c0ded0c81 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:27.213172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a14f5-e54b-476e-adda-c37c0ded0c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:27.248276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-3e4a14f5-e54b-476e-adda-c37c0ded0c81 None None] MFA Rules not processed for user `c63276ee73414ea6a7fde01990893bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:27.249539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3e4a14f5-e54b-476e-adda-c37c0ded0c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:27.321783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1103/2250] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:27.332967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7241325-2cbf-4c42-a2e6-938c167d5cbb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:27.336187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a7241325-2cbf-4c42-a2e6-938c167d5cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:27.415453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7241325-2cbf-4c42-a2e6-938c167d5cbb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:27.419905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca26eb-1955-4c1d-8a1a-3d271c7c65a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:27.420559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca26eb-1955-4c1d-8a1a-3d271c7c65a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:27.420870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca26eb-1955-4c1d-8a1a-3d271c7c65a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:27.421168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca26eb-1955-4c1d-8a1a-3d271c7c65a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:27.431341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca26eb-1955-4c1d-8a1a-3d271c7c65a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:27.432831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca26eb-1955-4c1d-8a1a-3d271c7c65a0 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:27.443342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1148/2251] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:33.195198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d3327-8d5d-4c4b-82a2-f1f16039d022 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:33.195773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d3327-8d5d-4c4b-82a2-f1f16039d022 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:33.195881 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d3327-8d5d-4c4b-82a2-f1f16039d022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:33.229816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b90d3327-8d5d-4c4b-82a2-f1f16039d022 None None] MFA Rules not processed for user `0b82b6f568bd41fa9d6b44facb766e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:33.231148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b90d3327-8d5d-4c4b-82a2-f1f16039d022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:33.308352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1104/2252] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:33.313608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca26eb-1955-4c1d-8a1a-3d271c7c65a0 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:33.320195 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:33.393303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca26eb-1955-4c1d-8a1a-3d271c7c65a0 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:33.399103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc530b-4785-4fb6-ae53-f03e29a2f0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:33.399727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc530b-4785-4fb6-ae53-f03e29a2f0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:33.400007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc530b-4785-4fb6-ae53-f03e29a2f0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:33.400277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc530b-4785-4fb6-ae53-f03e29a2f0c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:33.400960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc530b-4785-4fb6-ae53-f03e29a2f0c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:33.402942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc530b-4785-4fb6-ae53-f03e29a2f0c5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:33.561833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1149/2253] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:36:33 2022] POST /identity/v3/projects => generated 375 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:36:33.566097 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d3327-8d5d-4c4b-82a2-f1f16039d022 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:33.581117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d3327-8d5d-4c4b-82a2-f1f16039d022 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:33.614651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:33.648440 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4171e7-42d4-4ce8-8fb8-50af87fda469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:33.654361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4171e7-42d4-4ce8-8fb8-50af87fda469 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:33.654361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4171e7-42d4-4ce8-8fb8-50af87fda469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:33.654361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4171e7-42d4-4ce8-8fb8-50af87fda469 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:33.654361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4171e7-42d4-4ce8-8fb8-50af87fda469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:33.654361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4171e7-42d4-4ce8-8fb8-50af87fda469 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:33.752598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1105/2254] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:36:33 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:36:33.753989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc530b-4785-4fb6-ae53-f03e29a2f0c5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:33.766639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc530b-4785-4fb6-ae53-f03e29a2f0c5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:33.772308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9303b-7985-461d-80fb-c8a54d62f404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:33.773075 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9303b-7985-461d-80fb-c8a54d62f404 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:33.774122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9303b-7985-461d-80fb-c8a54d62f404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:33.774122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9303b-7985-461d-80fb-c8a54d62f404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:33.774810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9303b-7985-461d-80fb-c8a54d62f404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:33.776655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9303b-7985-461d-80fb-c8a54d62f404 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:33.800914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1150/2255] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:33.805022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4171e7-42d4-4ce8-8fb8-50af87fda469 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:33.817845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4171e7-42d4-4ce8-8fb8-50af87fda469 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:33.825408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1188ed-d62c-46af-81b3-53c635c23710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:33.826299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1188ed-d62c-46af-81b3-53c635c23710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:33.826503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1188ed-d62c-46af-81b3-53c635c23710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:33.826771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1188ed-d62c-46af-81b3-53c635c23710 admin admin] PATH_INFO: `/v3/projects/7609f88d3e834d29a09d17d3a0ba8614/users/6eee4d6ce51d4d2fa054ff69b0b61ea2/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:33.846683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1188ed-d62c-46af-81b3-53c635c23710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7609f88d3e834d29a09d17d3a0ba8614, user_id=6eee4d6ce51d4d2fa054ff69b0b61ea2, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:33.849742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1188ed-d62c-46af-81b3-53c635c23710 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:33.931699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1106/2256] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:33 2022] PUT /identity/v3/projects/7609f88d3e834d29a09d17d3a0ba8614/users/6eee4d6ce51d4d2fa054ff69b0b61ea2/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:36:33.935760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9303b-7985-461d-80fb-c8a54d62f404 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:33.948114 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9303b-7985-461d-80fb-c8a54d62f404 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:34.018083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8ff65-cc4c-4be3-85e8-facfebbbbe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:34.020773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8ff65-cc4c-4be3-85e8-facfebbbbe50 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:34.020773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8ff65-cc4c-4be3-85e8-facfebbbbe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:34.020773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8ff65-cc4c-4be3-85e8-facfebbbbe50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:34.020773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8ff65-cc4c-4be3-85e8-facfebbbbe50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:34.021598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8ff65-cc4c-4be3-85e8-facfebbbbe50 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:34.032765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1151/2257] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:36:34.037082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1188ed-d62c-46af-81b3-53c635c23710 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:34.048150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1188ed-d62c-46af-81b3-53c635c23710 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:34.054148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b34350-c2c4-4362-8b93-556805f0c765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:34.054874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b34350-c2c4-4362-8b93-556805f0c765 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:34.055277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b34350-c2c4-4362-8b93-556805f0c765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:34.055554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b34350-c2c4-4362-8b93-556805f0c765 admin admin] PATH_INFO: `/v3/projects/7609f88d3e834d29a09d17d3a0ba8614/users/6eee4d6ce51d4d2fa054ff69b0b61ea2/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:34.057535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b34350-c2c4-4362-8b93-556805f0c765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7609f88d3e834d29a09d17d3a0ba8614, user_id=6eee4d6ce51d4d2fa054ff69b0b61ea2, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:34.060314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b34350-c2c4-4362-8b93-556805f0c765 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:34.084326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1107/2258] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:36:34 2022] PUT /identity/v3/projects/7609f88d3e834d29a09d17d3a0ba8614/users/6eee4d6ce51d4d2fa054ff69b0b61ea2/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:36:41.027665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399b6b9-ca3e-45b0-9770-5c88f7ce06fb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.028271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399b6b9-ca3e-45b0-9770-5c88f7ce06fb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.028435 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399b6b9-ca3e-45b0-9770-5c88f7ce06fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.067166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6399b6b9-ca3e-45b0-9770-5c88f7ce06fb None None] MFA Rules not processed for user `6eee4d6ce51d4d2fa054ff69b0b61ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:41.069165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6399b6b9-ca3e-45b0-9770-5c88f7ce06fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:41.180520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:41.186248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1152/2259] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:41.212415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b34350-c2c4-4362-8b93-556805f0c765 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:41.216374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-04b34350-c2c4-4362-8b93-556805f0c765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:41.377737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b34350-c2c4-4362-8b93-556805f0c765 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:41.390335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb69d33-39f4-47dd-b595-2c3c9f845e10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:41.392476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb69d33-39f4-47dd-b595-2c3c9f845e10 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:41.393414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb69d33-39f4-47dd-b595-2c3c9f845e10 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:41.406004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb69d33-39f4-47dd-b595-2c3c9f845e10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:41.511024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb69d33-39f4-47dd-b595-2c3c9f845e10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:41.515055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb69d33-39f4-47dd-b595-2c3c9f845e10 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:41.534913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1108/2260] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:45.538771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399b6b9-ca3e-45b0-9770-5c88f7ce06fb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:45.606647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399b6b9-ca3e-45b0-9770-5c88f7ce06fb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:45.610943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9eb44e-a289-4f5c-81cd-4494ffc56fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:45.611507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9eb44e-a289-4f5c-81cd-4494ffc56fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:45.611795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9eb44e-a289-4f5c-81cd-4494ffc56fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:45.612051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9eb44e-a289-4f5c-81cd-4494ffc56fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:45.613792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-bb9eb44e-a289-4f5c-81cd-4494ffc56fe0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:45.631474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9eb44e-a289-4f5c-81cd-4494ffc56fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:45.632949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9eb44e-a289-4f5c-81cd-4494ffc56fe0 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:45.646109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1153/2261] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:50.107323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb69d33-39f4-47dd-b595-2c3c9f845e10 service nova] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:50.136678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:50.187138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb69d33-39f4-47dd-b595-2c3c9f845e10 service nova] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:50.193426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912303ff-7942-41bf-9d5d-f6f16aa44bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:50.194069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912303ff-7942-41bf-9d5d-f6f16aa44bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:50.194447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912303ff-7942-41bf-9d5d-f6f16aa44bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:50.194889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912303ff-7942-41bf-9d5d-f6f16aa44bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:50.197626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-912303ff-7942-41bf-9d5d-f6f16aa44bf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:50.217462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912303ff-7942-41bf-9d5d-f6f16aa44bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:50.219850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912303ff-7942-41bf-9d5d-f6f16aa44bf3 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:50.233631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1109/2262] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:36:52.074954 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9eb44e-a289-4f5c-81cd-4494ffc56fe0 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:52.167367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9eb44e-a289-4f5c-81cd-4494ffc56fe0 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:52.172288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:52.173015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:52.173292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:52.173564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] PATH_INFO: `/v3/users/c63276ee73414ea6a7fde01990893bfb` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:52.174894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c63276ee73414ea6a7fde01990893bfb)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:52.176871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:52.639950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c63276ee73414ea6a7fde01990893bfb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:52.827482 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c63276ee73414ea6a7fde01990893bfb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:52.842287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c63276ee73414ea6a7fde01990893bfb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:52.843023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c63276ee73414ea6a7fde01990893bfb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:52.850405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c63276ee73414ea6a7fde01990893bfb'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:52.856340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1154/2263] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:36:52 2022] DELETE /identity/v3/users/c63276ee73414ea6a7fde01990893bfb => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:36:52.861663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d592cd-9a9d-43b9-92cf-158b033edad1 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:52.862403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d592cd-9a9d-43b9-92cf-158b033edad1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:52.862403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d592cd-9a9d-43b9-92cf-158b033edad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:52.892599 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-00d592cd-9a9d-43b9-92cf-158b033edad1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:36:52.894125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-00d592cd-9a9d-43b9-92cf-158b033edad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:52.986413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1110/2264] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:36:52.997608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:52.999070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:53.034616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:36:53.084117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89131aa8-8602-44ac-99fe-518c250b1d01 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:53.088760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876bce3-f838-448a-9479-62c36c834d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:53.089240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876bce3-f838-448a-9479-62c36c834d19 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:53.089528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876bce3-f838-448a-9479-62c36c834d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:53.089852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876bce3-f838-448a-9479-62c36c834d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:53.100281 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876bce3-f838-448a-9479-62c36c834d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:53.101831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876bce3-f838-448a-9479-62c36c834d19 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:53.112709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1155/2265] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:36:53.744847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d592cd-9a9d-43b9-92cf-158b033edad1 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:36:53.744847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-00d592cd-9a9d-43b9-92cf-158b033edad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:36:53.761412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d592cd-9a9d-43b9-92cf-158b033edad1 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:36:53.768190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:36:53.769019 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:36:53.769405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:36:53.769783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] PATH_INFO: `/v3/projects/5137e6ff4dea4dea9b31f988865d68de` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:36:53.771936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5137e6ff4dea4dea9b31f988865d68de)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:36:53.775056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:36:53.956466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5137e6ff4dea4dea9b31f988865d68de', 'request_id': 'req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:53.964339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5137e6ff4dea4dea9b31f988865d68de', 'request_id': 'req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:54.113477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5137e6ff4dea4dea9b31f988865d68de', 'request_id': 'req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:54.127740 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] The token cache is being invalidate because project 5137e6ff4dea4dea9b31f988865d68de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:36:54.128115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:54.128829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:36:54.131291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1111/2266] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:36:53 2022] DELETE /identity/v3/projects/5137e6ff4dea4dea9b31f988865d68de => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:01.959905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876bce3-f838-448a-9479-62c36c834d19 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:01.961277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d876bce3-f838-448a-9479-62c36c834d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:02.047998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876bce3-f838-448a-9479-62c36c834d19 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:02.132520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10bc1b-b515-487f-9ef5-dbf12064f9d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:02.133576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10bc1b-b515-487f-9ef5-dbf12064f9d2 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:02.133957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10bc1b-b515-487f-9ef5-dbf12064f9d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:02.134365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10bc1b-b515-487f-9ef5-dbf12064f9d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:02.137167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5a10bc1b-b515-487f-9ef5-dbf12064f9d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:02.199800 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10bc1b-b515-487f-9ef5-dbf12064f9d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:02.201461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10bc1b-b515-487f-9ef5-dbf12064f9d2 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:02.206388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:02.212576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1156/2267] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:37:04.124568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:08.355766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:08.357966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:08.377373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3cfa9-3d07-41ef-b8a9-e213f5cdb872 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:08.385688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83379add-cd8a-448b-a498-a06156e05762 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:08.386336 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83379add-cd8a-448b-a498-a06156e05762 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:08.386608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83379add-cd8a-448b-a498-a06156e05762 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:08.386941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83379add-cd8a-448b-a498-a06156e05762 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:08.391480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-83379add-cd8a-448b-a498-a06156e05762 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:08.468072 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83379add-cd8a-448b-a498-a06156e05762 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:08.471217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83379add-cd8a-448b-a498-a06156e05762 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:08.480892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1112/2268] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:10.640292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10bc1b-b515-487f-9ef5-dbf12064f9d2 service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:10.655055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10bc1b-b515-487f-9ef5-dbf12064f9d2 service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:10.661371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:10.662140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:10.662411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:10.662686 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:10.664853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:10.737105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:10.740972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:10.750102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1157/2269] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:18.475712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=28 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:20.744954 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=28 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:31.652717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55586e3a-2cd9-474e-9929-3883449bc9a9 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:31.654189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55586e3a-2cd9-474e-9929-3883449bc9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:31.654624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55586e3a-2cd9-474e-9929-3883449bc9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:31.680266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-55586e3a-2cd9-474e-9929-3883449bc9a9 None None] MFA Rules not processed for user `0b82b6f568bd41fa9d6b44facb766e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:31.682050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-55586e3a-2cd9-474e-9929-3883449bc9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:31.763750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1113/2270] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:37:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:32.005161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:32.007526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:32.083739 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556ace7-0ab5-47b7-bcce-b9de63c50d8d service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:32.087713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:32.088328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:32.088771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:32.089082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:32.090953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:32.135335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55586e3a-2cd9-474e-9929-3883449bc9a9 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:32.137125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-55586e3a-2cd9-474e-9929-3883449bc9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:32.179619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:32.181125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:32.193013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1158/2271] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:32.212540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55586e3a-2cd9-474e-9929-3883449bc9a9 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:32.217426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de1f77-9b26-4d10-bdc5-71010b57ff6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:32.217957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de1f77-9b26-4d10-bdc5-71010b57ff6a service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:32.218459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de1f77-9b26-4d10-bdc5-71010b57ff6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:32.219234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de1f77-9b26-4d10-bdc5-71010b57ff6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:32.220833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e5de1f77-9b26-4d10-bdc5-71010b57ff6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:32.232968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de1f77-9b26-4d10-bdc5-71010b57ff6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:32.240313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de1f77-9b26-4d10-bdc5-71010b57ff6a service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:32.247969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1114/2272] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:37:33.417026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:33.419183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:33.514718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1306aa-efd7-4128-adf9-c4a05e019a1e service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:33.522181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:33.522884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.523232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.523619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.525927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:33.539854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:33.541541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:33.554288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1159/2273] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:33.679735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea64f9-ea4f-43ba-9104-6c643d128475 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:33.680275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea64f9-ea4f-43ba-9104-6c643d128475 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:33.680385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea64f9-ea4f-43ba-9104-6c643d128475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:33.702856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b6ea64f9-ea4f-43ba-9104-6c643d128475 None None] MFA Rules not processed for user `2715cf2a5129439c83a5347bb963f1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:33.704135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b6ea64f9-ea4f-43ba-9104-6c643d128475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:33.712145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1115/2274] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:37:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:36.868504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:36.872428 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:36.886249 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc933e1-847e-4fce-9c2a-ed6a17e6a1b1 service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:36.892298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa359-81aa-4ad4-a978-b5817a9f04d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:36.892968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa359-81aa-4ad4-a978-b5817a9f04d2 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:36.893242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa359-81aa-4ad4-a978-b5817a9f04d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:36.893629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa359-81aa-4ad4-a978-b5817a9f04d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:36.896602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cadaa359-81aa-4ad4-a978-b5817a9f04d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:36.911237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaa359-81aa-4ad4-a978-b5817a9f04d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:36.912982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaa359-81aa-4ad4-a978-b5817a9f04d2 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:36.929693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1160/2275] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:37:40.974629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea64f9-ea4f-43ba-9104-6c643d128475 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:40.980747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b6ea64f9-ea4f-43ba-9104-6c643d128475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:41.014491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea64f9-ea4f-43ba-9104-6c643d128475 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:41.022323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae22e3-00df-43c5-9e63-9f8ce3df04c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:41.023323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae22e3-00df-43c5-9e63-9f8ce3df04c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:41.023560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae22e3-00df-43c5-9e63-9f8ce3df04c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:41.023838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae22e3-00df-43c5-9e63-9f8ce3df04c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:41.025804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-71ae22e3-00df-43c5-9e63-9f8ce3df04c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:41.043416 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae22e3-00df-43c5-9e63-9f8ce3df04c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:41.044961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae22e3-00df-43c5-9e63-9f8ce3df04c6 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:41.069146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1116/2276] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:43.781175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa359-81aa-4ad4-a978-b5817a9f04d2 service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:43.815017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa359-81aa-4ad4-a978-b5817a9f04d2 service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:43.822255 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:43.823106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:43.823762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:43.824341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:43.827375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:43.899970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae22e3-00df-43c5-9e63-9f8ce3df04c6 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:43.924096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae22e3-00df-43c5-9e63-9f8ce3df04c6 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:43.931668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0a274-6913-421a-aa7a-c9044dda90f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:43.932546 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0a274-6913-421a-aa7a-c9044dda90f6 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:43.933045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0a274-6913-421a-aa7a-c9044dda90f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:43.933509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0a274-6913-421a-aa7a-c9044dda90f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:43.936745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:43.938651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-33f0a274-6913-421a-aa7a-c9044dda90f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:43.966105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0a274-6913-421a-aa7a-c9044dda90f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:43.970526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0a274-6913-421a-aa7a-c9044dda90f6 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:43.996363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1117/2277] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:44.030378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:44.031833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:44.046024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1161/2278] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:37:44.442222 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa9882-f4fd-497c-bfe6-262ef3334085 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:44.442524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa9882-f4fd-497c-bfe6-262ef3334085 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:44.443002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa9882-f4fd-497c-bfe6-262ef3334085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:44.465045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-14aa9882-f4fd-497c-bfe6-262ef3334085 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:44.466678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-14aa9882-f4fd-497c-bfe6-262ef3334085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:44.566481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1118/2279] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:37:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:37:44.640180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:44.641873 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:44.755409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efd9d7-76a8-41fc-b19e-fd4d188aa783 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:44.761155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18e428-51a0-4de9-9b39-f4473dbbf1c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:44.761836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18e428-51a0-4de9-9b39-f4473dbbf1c7 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:44.762118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18e428-51a0-4de9-9b39-f4473dbbf1c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:44.762381 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18e428-51a0-4de9-9b39-f4473dbbf1c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:44.773627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18e428-51a0-4de9-9b39-f4473dbbf1c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:44.776853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18e428-51a0-4de9-9b39-f4473dbbf1c7 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:44.807908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1162/2280] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:37:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:37:45.240196 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa9882-f4fd-497c-bfe6-262ef3334085 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:45.260505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa9882-f4fd-497c-bfe6-262ef3334085 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:45.272766 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e40685-b97c-4d76-affb-3096c6e40369 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:45.273728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e40685-b97c-4d76-affb-3096c6e40369 service placement] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:45.274176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e40685-b97c-4d76-affb-3096c6e40369 service placement] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:45.274657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e40685-b97c-4d76-affb-3096c6e40369 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:45.282854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-65e40685-b97c-4d76-affb-3096c6e40369 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:45.305622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e40685-b97c-4d76-affb-3096c6e40369 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:45.307559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e40685-b97c-4d76-affb-3096c6e40369 service placement] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:45.325528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1119/2281] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:37:45.609127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18e428-51a0-4de9-9b39-f4473dbbf1c7 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:45.611572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8f18e428-51a0-4de9-9b39-f4473dbbf1c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:45.659138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18e428-51a0-4de9-9b39-f4473dbbf1c7 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:45.659138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:45.659138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:45.659138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:45.659555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] PATH_INFO: `/v3/users/5055ddeab8314c93bb2a664f9d1a7f3a` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:45.659555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5055ddeab8314c93bb2a664f9d1a7f3a)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:45.664960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:46.794543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5055ddeab8314c93bb2a664f9d1a7f3a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:46.917880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5055ddeab8314c93bb2a664f9d1a7f3a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:46.969794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5055ddeab8314c93bb2a664f9d1a7f3a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:46.971911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5055ddeab8314c93bb2a664f9d1a7f3a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:46.984697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b1a3ea35-eba7-44d4-a43c-ff7819b292ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5055ddeab8314c93bb2a664f9d1a7f3a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:46.995950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1163/2282] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:37:45 2022] DELETE /identity/v3/users/5055ddeab8314c93bb2a664f9d1a7f3a => generated 0 bytes in 1384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:47.559330 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e40685-b97c-4d76-affb-3096c6e40369 service placement] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:47.579676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e40685-b97c-4d76-affb-3096c6e40369 service placement] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:47.639935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:47.745889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:47.746738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:47.747092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:47.748006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] PATH_INFO: `/v3/projects/f910065fc9da40798c7f3d4366dab7c6` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:47.749194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f910065fc9da40798c7f3d4366dab7c6)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:47.751307 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:48.023376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f910065fc9da40798c7f3d4366dab7c6', 'request_id': 'req-323db85d-96d1-4867-b17b-3274fc64a323'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.038587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f910065fc9da40798c7f3d4366dab7c6', 'request_id': 'req-323db85d-96d1-4867-b17b-3274fc64a323'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.135290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f910065fc9da40798c7f3d4366dab7c6', 'request_id': 'req-323db85d-96d1-4867-b17b-3274fc64a323'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.163383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] The token cache is being invalidate because project f910065fc9da40798c7f3d4366dab7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:37:48.163714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.169339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-323db85d-96d1-4867-b17b-3274fc64a323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:37:48.180668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1120/2283] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:37:47 2022] DELETE /identity/v3/projects/f910065fc9da40798c7f3d4366dab7c6 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:48.182914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf96ff8-c738-45e4-81d7-43a65b7fdbf5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:48.182914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf96ff8-c738-45e4-81d7-43a65b7fdbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:48.182914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf96ff8-c738-45e4-81d7-43a65b7fdbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:48.213232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1bf96ff8-c738-45e4-81d7-43a65b7fdbf5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:48.215343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1bf96ff8-c738-45e4-81d7-43a65b7fdbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:48.250379 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:48.319550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1164/2284] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:48.338693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc43730-dc4b-43b9-acf1-1b13031197f2 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:48.339246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc43730-dc4b-43b9-acf1-1b13031197f2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:48.339655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc43730-dc4b-43b9-acf1-1b13031197f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:48.389698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-4fc43730-dc4b-43b9-acf1-1b13031197f2 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:48.391146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4fc43730-dc4b-43b9-acf1-1b13031197f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:48.407142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1121/2285] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:37:48.412043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf96ff8-c738-45e4-81d7-43a65b7fdbf5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:48.428027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf96ff8-c738-45e4-81d7-43a65b7fdbf5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:48.435625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e529-70db-4809-bfa2-22cfc51e07fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:48.451799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e529-70db-4809-bfa2-22cfc51e07fa admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:48.451799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e529-70db-4809-bfa2-22cfc51e07fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:48.451799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5e529-70db-4809-bfa2-22cfc51e07fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:48.451799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5e529-70db-4809-bfa2-22cfc51e07fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:37:48.451799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5e529-70db-4809-bfa2-22cfc51e07fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:48.451799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5e529-70db-4809-bfa2-22cfc51e07fa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:48.468300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1165/2286] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:37:48.474205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc43730-dc4b-43b9-acf1-1b13031197f2 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:48.485033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc43730-dc4b-43b9-acf1-1b13031197f2 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:48.497988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba18f97-447b-4d6f-bcfe-e0f4ffb1a7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:48.499101 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba18f97-447b-4d6f-bcfe-e0f4ffb1a7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:48.499796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba18f97-447b-4d6f-bcfe-e0f4ffb1a7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:48.500525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba18f97-447b-4d6f-bcfe-e0f4ffb1a7b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:48.501941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba18f97-447b-4d6f-bcfe-e0f4ffb1a7b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:48.510700 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba18f97-447b-4d6f-bcfe-e0f4ffb1a7b9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:48.842274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1122/2287] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:37:48 2022] POST /identity/v3/projects => generated 373 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:48.843153 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e529-70db-4809-bfa2-22cfc51e07fa admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:48.850962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5e529-70db-4809-bfa2-22cfc51e07fa admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:48.925821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad33ac-1f47-4522-b8dc-e9f3cce38c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:48.926473 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad33ac-1f47-4522-b8dc-e9f3cce38c21 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:48.926852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad33ac-1f47-4522-b8dc-e9f3cce38c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:48.927121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad33ac-1f47-4522-b8dc-e9f3cce38c21 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:48.927762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad33ac-1f47-4522-b8dc-e9f3cce38c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:48.929825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad33ac-1f47-4522-b8dc-e9f3cce38c21 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:49.665532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1166/2288] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:37:48 2022] POST /identity/v3/users => generated 287 bytes in 828 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:37:49.669525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba18f97-447b-4d6f-bcfe-e0f4ffb1a7b9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:49.681875 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba18f97-447b-4d6f-bcfe-e0f4ffb1a7b9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:49.686637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e24db8-d025-4090-8532-cb063245918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:49.687251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e24db8-d025-4090-8532-cb063245918c admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:49.687535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e24db8-d025-4090-8532-cb063245918c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:49.687863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e24db8-d025-4090-8532-cb063245918c admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:49.688512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e24db8-d025-4090-8532-cb063245918c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:49.690243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e24db8-d025-4090-8532-cb063245918c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:49.702466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1123/2289] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:49.713061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad33ac-1f47-4522-b8dc-e9f3cce38c21 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:49.726994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad33ac-1f47-4522-b8dc-e9f3cce38c21 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:49.731967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f529bee-2492-4622-a846-e3fda3bd31f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:49.732582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f529bee-2492-4622-a846-e3fda3bd31f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:49.732889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f529bee-2492-4622-a846-e3fda3bd31f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:49.733179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f529bee-2492-4622-a846-e3fda3bd31f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:49.733828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f529bee-2492-4622-a846-e3fda3bd31f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:49.735486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f529bee-2492-4622-a846-e3fda3bd31f9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:49.748418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1167/2290] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:37:49.753129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e24db8-d025-4090-8532-cb063245918c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:49.770547 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e24db8-d025-4090-8532-cb063245918c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:49.782386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d72813-984d-489d-8fd7-45bd54cba5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:49.783252 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d72813-984d-489d-8fd7-45bd54cba5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:49.783636 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d72813-984d-489d-8fd7-45bd54cba5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:49.784062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d72813-984d-489d-8fd7-45bd54cba5a1 admin admin] PATH_INFO: `/v3/projects/31998483af514a1a8f87ab761d5cf3ba/users/7ac5dffd3204413a8834d05db267c3ab/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:49.802523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d72813-984d-489d-8fd7-45bd54cba5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31998483af514a1a8f87ab761d5cf3ba, user_id=7ac5dffd3204413a8834d05db267c3ab, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:49.805941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d72813-984d-489d-8fd7-45bd54cba5a1 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:49.908430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1124/2291] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:37:49 2022] PUT /identity/v3/projects/31998483af514a1a8f87ab761d5cf3ba/users/7ac5dffd3204413a8834d05db267c3ab/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:37:49.921271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181b344-e36a-44db-a7a5-e6daf2fd1bfa None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:49.921271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181b344-e36a-44db-a7a5-e6daf2fd1bfa None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:49.921271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181b344-e36a-44db-a7a5-e6daf2fd1bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:49.947087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6181b344-e36a-44db-a7a5-e6daf2fd1bfa None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:37:49.948348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6181b344-e36a-44db-a7a5-e6daf2fd1bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:50.045662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1168/2292] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:37:50.070730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d72813-984d-489d-8fd7-45bd54cba5a1 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:37:50.087510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-67d72813-984d-489d-8fd7-45bd54cba5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:37:50.132324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:37:50.191943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d72813-984d-489d-8fd7-45bd54cba5a1 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:37:50.197872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38aa3f-2be5-4c75-a53c-743d699f8de6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:37:50.198519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38aa3f-2be5-4c75-a53c-743d699f8de6 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:37:50.198795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38aa3f-2be5-4c75-a53c-743d699f8de6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:37:50.199066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38aa3f-2be5-4c75-a53c-743d699f8de6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:37:50.236168 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38aa3f-2be5-4c75-a53c-743d699f8de6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:37:50.238817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38aa3f-2be5-4c75-a53c-743d699f8de6 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:37:50.276548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1125/2293] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:02.021819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727857f-39e0-439b-9d3e-982bd511e994 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:02.022461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727857f-39e0-439b-9d3e-982bd511e994 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:02.022562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727857f-39e0-439b-9d3e-982bd511e994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:02.057120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4727857f-39e0-439b-9d3e-982bd511e994 None None] MFA Rules not processed for user `7ac5dffd3204413a8834d05db267c3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:38:02.058807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4727857f-39e0-439b-9d3e-982bd511e994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:02.086009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:02.157133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1169/2294] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:38:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:38:02.164147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38aa3f-2be5-4c75-a53c-743d699f8de6 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:02.185271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38aa3f-2be5-4c75-a53c-743d699f8de6 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:02.191454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152db6a-ba93-4e5d-afb0-50956e1b4918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:02.192467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152db6a-ba93-4e5d-afb0-50956e1b4918 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:02.192820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152db6a-ba93-4e5d-afb0-50956e1b4918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:02.193251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152db6a-ba93-4e5d-afb0-50956e1b4918 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:02.194012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152db6a-ba93-4e5d-afb0-50956e1b4918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:02.195961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152db6a-ba93-4e5d-afb0-50956e1b4918 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:02.634232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1126/2295] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:38:02 2022] POST /identity/v3/projects => generated 371 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:38:02.648753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727857f-39e0-439b-9d3e-982bd511e994 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:02.681295 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727857f-39e0-439b-9d3e-982bd511e994 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:02.818794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ca982-9b85-4523-8ac3-561ccb7061dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:02.819434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca982-9b85-4523-8ac3-561ccb7061dd admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:02.822705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca982-9b85-4523-8ac3-561ccb7061dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:02.823541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ca982-9b85-4523-8ac3-561ccb7061dd admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:02.828656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ca982-9b85-4523-8ac3-561ccb7061dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:02.834525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ca982-9b85-4523-8ac3-561ccb7061dd admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:03.007505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1170/2296] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:38:02 2022] POST /identity/v3/users => generated 292 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:38:03.011739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152db6a-ba93-4e5d-afb0-50956e1b4918 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:03.024352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152db6a-ba93-4e5d-afb0-50956e1b4918 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:03.036496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08580029-1419-4941-b9cf-b8b4f35f1799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:03.039385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08580029-1419-4941-b9cf-b8b4f35f1799 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:03.040230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08580029-1419-4941-b9cf-b8b4f35f1799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:03.041074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08580029-1419-4941-b9cf-b8b4f35f1799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:03.042095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08580029-1419-4941-b9cf-b8b4f35f1799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:03.045896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08580029-1419-4941-b9cf-b8b4f35f1799 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:03.066328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ca982-9b85-4523-8ac3-561ccb7061dd admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:03.077651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1127/2297] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:38:03.086688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ca982-9b85-4523-8ac3-561ccb7061dd admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:03.110923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1135a6-12b3-4177-8dbd-7b94a428730a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:03.111256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1135a6-12b3-4177-8dbd-7b94a428730a admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:03.111884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1135a6-12b3-4177-8dbd-7b94a428730a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:03.112509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1135a6-12b3-4177-8dbd-7b94a428730a admin admin] PATH_INFO: `/v3/projects/2157fe55e5744b6bbcb6afecbff2caf2/users/353863ab3a164b9fa49644358223096a/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:03.164309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1135a6-12b3-4177-8dbd-7b94a428730a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2157fe55e5744b6bbcb6afecbff2caf2, user_id=353863ab3a164b9fa49644358223096a, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:03.170527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1135a6-12b3-4177-8dbd-7b94a428730a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:03.326336 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1171/2298] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:38:03 2022] PUT /identity/v3/projects/2157fe55e5744b6bbcb6afecbff2caf2/users/353863ab3a164b9fa49644358223096a/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:38:03.337284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08580029-1419-4941-b9cf-b8b4f35f1799 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:03.353709 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08580029-1419-4941-b9cf-b8b4f35f1799 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:03.454514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07807592-0754-4222-ab61-d19a21a665eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:03.456260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07807592-0754-4222-ab61-d19a21a665eb admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:03.456742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07807592-0754-4222-ab61-d19a21a665eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:03.457286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07807592-0754-4222-ab61-d19a21a665eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:03.458738 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07807592-0754-4222-ab61-d19a21a665eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:03.462973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07807592-0754-4222-ab61-d19a21a665eb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:03.478470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1128/2299] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:38:03.488632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1135a6-12b3-4177-8dbd-7b94a428730a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:03.521091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1135a6-12b3-4177-8dbd-7b94a428730a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:03.537483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722754ed-06eb-49c3-9123-61d2f75e7caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:03.539173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722754ed-06eb-49c3-9123-61d2f75e7caa admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:03.547842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722754ed-06eb-49c3-9123-61d2f75e7caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:03.548520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722754ed-06eb-49c3-9123-61d2f75e7caa admin admin] PATH_INFO: `/v3/projects/2157fe55e5744b6bbcb6afecbff2caf2/users/353863ab3a164b9fa49644358223096a/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:03.552004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722754ed-06eb-49c3-9123-61d2f75e7caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2157fe55e5744b6bbcb6afecbff2caf2, user_id=353863ab3a164b9fa49644358223096a, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:03.559170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722754ed-06eb-49c3-9123-61d2f75e7caa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:03.594484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1172/2300] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:38:03 2022] PUT /identity/v3/projects/2157fe55e5744b6bbcb6afecbff2caf2/users/353863ab3a164b9fa49644358223096a/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:38:09.092332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07807592-0754-4222-ab61-d19a21a665eb admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:09.097086 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-07807592-0754-4222-ab61-d19a21a665eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:09.196503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07807592-0754-4222-ab61-d19a21a665eb admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:09.276779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:09.350865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097362a-2a37-4bd1-b614-8313ddb570d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:09.351509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097362a-2a37-4bd1-b614-8313ddb570d3 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:09.351790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097362a-2a37-4bd1-b614-8313ddb570d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:09.352062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097362a-2a37-4bd1-b614-8313ddb570d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:09.353972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7097362a-2a37-4bd1-b614-8313ddb570d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:09.373754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097362a-2a37-4bd1-b614-8313ddb570d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:09.376073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097362a-2a37-4bd1-b614-8313ddb570d3 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:09.422774 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1129/2301] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:12.013048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722754ed-06eb-49c3-9123-61d2f75e7caa admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:12.021048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-722754ed-06eb-49c3-9123-61d2f75e7caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:12.176077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722754ed-06eb-49c3-9123-61d2f75e7caa admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:12.180300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:12.181155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:12.182881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:12.182881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:12.184009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:12.223648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:12.274496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:12.276432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:12.288338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1173/2302] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:14.613103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2463b8-2c3a-4884-8a30-11076c9730fb None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:14.614332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2463b8-2c3a-4884-8a30-11076c9730fb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:14.614985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2463b8-2c3a-4884-8a30-11076c9730fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:14.677814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-4a2463b8-2c3a-4884-8a30-11076c9730fb None None] MFA Rules not processed for user `353863ab3a164b9fa49644358223096a`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:38:14.679472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4a2463b8-2c3a-4884-8a30-11076c9730fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:14.801496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1130/2303] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:38:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:38:14.820541 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:14.834202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:14.875990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6095d-466a-40da-87f6-fa5a25d1b15d service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:14.887289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47580a10-0384-4190-b40b-1c160808e273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:14.889319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47580a10-0384-4190-b40b-1c160808e273 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:14.890061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47580a10-0384-4190-b40b-1c160808e273 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:14.890799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47580a10-0384-4190-b40b-1c160808e273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:14.920978 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2463b8-2c3a-4884-8a30-11076c9730fb None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:14.941203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2463b8-2c3a-4884-8a30-11076c9730fb None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:14.945917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd5a55-6854-406b-a954-0a8c07548768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:14.946729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd5a55-6854-406b-a954-0a8c07548768 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:14.947338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd5a55-6854-406b-a954-0a8c07548768 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:14.947772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd5a55-6854-406b-a954-0a8c07548768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:14.952315 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ddcd5a55-6854-406b-a954-0a8c07548768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:14.976257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd5a55-6854-406b-a954-0a8c07548768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:14.979584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd5a55-6854-406b-a954-0a8c07548768 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:14.994413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47580a10-0384-4190-b40b-1c160808e273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:14.998925 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47580a10-0384-4190-b40b-1c160808e273 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:15.007646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1131/2304] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:38:15.016523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1174/2305] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:38:15.565532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522e7fd-41b7-41fa-b049-b2a1e0849f7f None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:15.565965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522e7fd-41b7-41fa-b049-b2a1e0849f7f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:15.566291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522e7fd-41b7-41fa-b049-b2a1e0849f7f None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:15.569160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1132/2306] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:38:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 11:38:15.576511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e967-ca2d-46be-87bd-1a37d2780b01 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:15.576927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e967-ca2d-46be-87bd-1a37d2780b01 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:15.577293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e967-ca2d-46be-87bd-1a37d2780b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:15.625574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d931e967-ca2d-46be-87bd-1a37d2780b01 None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:38:15.627759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d931e967-ca2d-46be-87bd-1a37d2780b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:15.645437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1175/2307] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:38:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:38:15.663226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522e7fd-41b7-41fa-b049-b2a1e0849f7f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:15.664295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c522e7fd-41b7-41fa-b049-b2a1e0849f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:15.759453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522e7fd-41b7-41fa-b049-b2a1e0849f7f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:15.764655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6317b4-c343-49fa-acec-b1ba1d9013f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ac781cf2d84d95a789724e74e12acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:15.765944 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6317b4-c343-49fa-acec-b1ba1d9013f0 service glance] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:15.766541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6317b4-c343-49fa-acec-b1ba1d9013f0 service glance] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:15.768239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6317b4-c343-49fa-acec-b1ba1d9013f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:15.784653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6317b4-c343-49fa-acec-b1ba1d9013f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:15.786701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6317b4-c343-49fa-acec-b1ba1d9013f0 service glance] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:15.832134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1133/2308] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:16.135233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf738a-06f5-4d36-91d5-09594dc564a5 None None] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:16.136307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf738a-06f5-4d36-91d5-09594dc564a5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:16.137001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf738a-06f5-4d36-91d5-09594dc564a5 None None] PATH_INFO: `/` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:16.138378 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1176/2309] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:38:16.144586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3effd-7b70-474e-81ef-25740993b036 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:16.146169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3effd-7b70-474e-81ef-25740993b036 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:16.152424 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3effd-7b70-474e-81ef-25740993b036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:16.187849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c6d3effd-7b70-474e-81ef-25740993b036 None None] MFA Rules not processed for user `372e96bd3361434e9169b783f0c9ea13`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:38:16.190805 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c6d3effd-7b70-474e-81ef-25740993b036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:16.205657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1134/2310] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:38:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:38:16.224711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf738a-06f5-4d36-91d5-09594dc564a5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:16.255120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf738a-06f5-4d36-91d5-09594dc564a5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:16.262826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af0b37-32ce-4bfc-b997-c0dfbe1bc6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:16.283445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af0b37-32ce-4bfc-b997-c0dfbe1bc6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:16.284537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af0b37-32ce-4bfc-b997-c0dfbe1bc6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:16.288415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af0b37-32ce-4bfc-b997-c0dfbe1bc6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:16.322487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af0b37-32ce-4bfc-b997-c0dfbe1bc6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:16.326988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af0b37-32ce-4bfc-b997-c0dfbe1bc6c8 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:16.364117 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1177/2311] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:16.505996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdb4c7-1cfc-4e7b-ba14-726afb8f44b0 None None] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:16.506741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdb4c7-1cfc-4e7b-ba14-726afb8f44b0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:16.507327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdb4c7-1cfc-4e7b-ba14-726afb8f44b0 None None] PATH_INFO: `/` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:16.509610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1135/2312] 10.222.0.84 () {54 vars in 921 bytes} [Wed Aug 24 11:38:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 11:38:16.521777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aec2e2-e67f-4483-9613-989f77279aa2 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:16.525915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aec2e2-e67f-4483-9613-989f77279aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:16.525915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aec2e2-e67f-4483-9613-989f77279aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:16.588639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f9aec2e2-e67f-4483-9613-989f77279aa2 None None] MFA Rules not processed for user `f58a559951c94d68951da78c82341a00`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:38:16.590425 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f9aec2e2-e67f-4483-9613-989f77279aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:16.604958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1178/2313] 10.222.0.84 () {58 vars in 1018 bytes} [Wed Aug 24 11:38:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:38:16.636585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdb4c7-1cfc-4e7b-ba14-726afb8f44b0 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:16.671799 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdb4c7-1cfc-4e7b-ba14-726afb8f44b0 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:16.679922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9c110-37aa-4173-84ec-bdfbf0b3d3e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:16.680841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9c110-37aa-4173-84ec-bdfbf0b3d3e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:16.682787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9c110-37aa-4173-84ec-bdfbf0b3d3e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:16.682787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9c110-37aa-4173-84ec-bdfbf0b3d3e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:16.698965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9c110-37aa-4173-84ec-bdfbf0b3d3e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:16.700994 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9c110-37aa-4173-84ec-bdfbf0b3d3e9 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:16.715095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1136/2314] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:19.130310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ef994-c805-492e-b596-ddc6fd499dbb None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:19.130769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ef994-c805-492e-b596-ddc6fd499dbb None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:19.130985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ef994-c805-492e-b596-ddc6fd499dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:19.154146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ad3ef994-c805-492e-b596-ddc6fd499dbb None None] MFA Rules not processed for user `7ac5dffd3204413a8834d05db267c3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:38:19.155426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ad3ef994-c805-492e-b596-ddc6fd499dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:19.164131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1179/2315] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:38:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:38:22.980421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f3340-b82c-4ac8-8059-0fbcf48181ba None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:22.981020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f3340-b82c-4ac8-8059-0fbcf48181ba None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:22.981118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f3340-b82c-4ac8-8059-0fbcf48181ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:23.004747 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-2c3f3340-b82c-4ac8-8059-0fbcf48181ba None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:38:23.005921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2c3f3340-b82c-4ac8-8059-0fbcf48181ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:23.019368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:23.100500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1137/2316] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:38:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:38:23.115757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ef994-c805-492e-b596-ddc6fd499dbb None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:23.117402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ad3ef994-c805-492e-b596-ddc6fd499dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:23.221083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ef994-c805-492e-b596-ddc6fd499dbb None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:23.226465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b77640-b316-46f9-ba1a-36ae699b8e45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:23.227078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b77640-b316-46f9-ba1a-36ae699b8e45 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:23.227355 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b77640-b316-46f9-ba1a-36ae699b8e45 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:23.227622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b77640-b316-46f9-ba1a-36ae699b8e45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:23.237354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b77640-b316-46f9-ba1a-36ae699b8e45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:23.239032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b77640-b316-46f9-ba1a-36ae699b8e45 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:23.248432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1180/2317] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:38:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:38:29.670829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f3340-b82c-4ac8-8059-0fbcf48181ba None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:29.693667 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f3340-b82c-4ac8-8059-0fbcf48181ba None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:29.705886 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea9edd-c129-459b-b180-d93a833d3ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:29.707593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea9edd-c129-459b-b180-d93a833d3ec3 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:29.708183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea9edd-c129-459b-b180-d93a833d3ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:29.708756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea9edd-c129-459b-b180-d93a833d3ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:29.712164 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-44ea9edd-c129-459b-b180-d93a833d3ec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:29.814490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea9edd-c129-459b-b180-d93a833d3ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:29.819539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea9edd-c129-459b-b180-d93a833d3ec3 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:29.841152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1138/2318] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:38:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:38:33.245120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:39.831869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=45 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:38:53.520007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f0e3-d2d2-4fd7-abfc-e6a01894521d None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:53.520007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f0e3-d2d2-4fd7-abfc-e6a01894521d None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:53.520733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f0e3-d2d2-4fd7-abfc-e6a01894521d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:53.549907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4c08f0e3-d2d2-4fd7-abfc-e6a01894521d None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:38:53.553456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4c08f0e3-d2d2-4fd7-abfc-e6a01894521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:38:53.568226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1181/2319] 10.222.0.84 () {58 vars in 967 bytes} [Wed Aug 24 11:38:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:38:53.594153 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea9edd-c129-459b-b180-d93a833d3ec3 service nova] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:38:53.606502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea9edd-c129-459b-b180-d93a833d3ec3 service nova] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:38:53.612379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e8836-8648-4353-b468-856a05967281 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:38:53.613228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e8836-8648-4353-b468-856a05967281 service swift] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:38:53.613581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e8836-8648-4353-b468-856a05967281 service swift] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:38:53.613972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e8836-8648-4353-b468-856a05967281 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:38:53.625575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e8836-8648-4353-b468-856a05967281 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:38:53.626392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e8836-8648-4353-b468-856a05967281 service swift] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:38:53.635920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1139/2320] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:38:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:39:03.540160 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:39:03.632485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=6 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:39:20.632010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f0e3-d2d2-4fd7-abfc-e6a01894521d None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:39:20.642392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f0e3-d2d2-4fd7-abfc-e6a01894521d None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:39:20.646304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0a723-02c2-4e24-bd69-892e06a109c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:39:20.646932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0a723-02c2-4e24-bd69-892e06a109c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:39:20.647209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0a723-02c2-4e24-bd69-892e06a109c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:39:20.647474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0a723-02c2-4e24-bd69-892e06a109c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:39:20.658468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0a723-02c2-4e24-bd69-892e06a109c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:39:20.659925 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0a723-02c2-4e24-bd69-892e06a109c7 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:39:20.684492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1182/2321] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:39:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:39:21.402989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff08f2-d9cb-499f-9883-a5e4f118ae20 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:39:21.403769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff08f2-d9cb-499f-9883-a5e4f118ae20 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:39:21.403945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff08f2-d9cb-499f-9883-a5e4f118ae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:39:21.428917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a9ff08f2-d9cb-499f-9883-a5e4f118ae20 None None] MFA Rules not processed for user `2d5bfffc316f481183093e386a326946`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:39:21.430969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a9ff08f2-d9cb-499f-9883-a5e4f118ae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:39:21.444679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1140/2322] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:39:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:39:30.672111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=6 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:39:31.420722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:13.169677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0a723-02c2-4e24-bd69-892e06a109c7 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:40:13.171871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-58b0a723-02c2-4e24-bd69-892e06a109c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:13.413491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0a723-02c2-4e24-bd69-892e06a109c7 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:40:13.422052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b583c-7304-4705-a900-a0f82b0f38c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:40:13.422758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b583c-7304-4705-a900-a0f82b0f38c9 service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:13.423353 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b583c-7304-4705-a900-a0f82b0f38c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:13.423783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b583c-7304-4705-a900-a0f82b0f38c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:13.426674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c28b583c-7304-4705-a900-a0f82b0f38c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:13.448838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b583c-7304-4705-a900-a0f82b0f38c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:40:13.451426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b583c-7304-4705-a900-a0f82b0f38c9 service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:40:13.469916 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1183/2323] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:40:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:40:23.462667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=63 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:31.283187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd635c-d6d7-4b71-94a4-a0d60a701d8f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:31.284336 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd635c-d6d7-4b71-94a4-a0d60a701d8f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:31.284336 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd635c-d6d7-4b71-94a4-a0d60a701d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:31.308809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-8afd635c-d6d7-4b71-94a4-a0d60a701d8f None None] MFA Rules not processed for user `4a80d0127ee349d69f0ed949bcb4c919`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:40:31.310033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8afd635c-d6d7-4b71-94a4-a0d60a701d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:31.373911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1141/2324] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:40:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:40:31.597974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd515-ece5-44e9-ad0e-abe6f017e329 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:40:31.598462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd515-ece5-44e9-ad0e-abe6f017e329 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:40:31.598576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd515-ece5-44e9-ad0e-abe6f017e329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:40:31.618889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-47fdd515-ece5-44e9-ad0e-abe6f017e329 None None] MFA Rules not processed for user `2715cf2a5129439c83a5347bb963f1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:40:31.620116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-47fdd515-ece5-44e9-ad0e-abe6f017e329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:40:31.689140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1184/2325] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:40:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:40:41.366850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=25 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:40:41.682325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=25 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:11.397240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd635c-d6d7-4b71-94a4-a0d60a701d8f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:11.424558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd635c-d6d7-4b71-94a4-a0d60a701d8f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:11.428591 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:11.429241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:11.429525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:11.429818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:11.431625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:11.538631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:11.540092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:11.562337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1142/2326] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:41:15.869303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f62c8-d683-48be-bed3-acd39b2f1f44 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:15.869903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f62c8-d683-48be-bed3-acd39b2f1f44 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:15.869903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f62c8-d683-48be-bed3-acd39b2f1f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:15.893412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-ad2f62c8-d683-48be-bed3-acd39b2f1f44 None None] MFA Rules not processed for user `0b82b6f568bd41fa9d6b44facb766e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:15.894977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ad2f62c8-d683-48be-bed3-acd39b2f1f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:15.993929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1185/2327] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:16.107638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:16.109200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:16.128887 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31145ba-754c-40c6-849a-fb627d6127d8 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:16.135651 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5feb96-6997-4189-9554-7eb075c55cc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:16.136826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5feb96-6997-4189-9554-7eb075c55cc0 service placement] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:16.137121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5feb96-6997-4189-9554-7eb075c55cc0 service placement] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:16.139268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5feb96-6997-4189-9554-7eb075c55cc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:16.139268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9f5feb96-6997-4189-9554-7eb075c55cc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:16.151852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5feb96-6997-4189-9554-7eb075c55cc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:16.153520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5feb96-6997-4189-9554-7eb075c55cc0 service placement] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:16.169653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1143/2328] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:41:16.288269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f62c8-d683-48be-bed3-acd39b2f1f44 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:16.290485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ad2f62c8-d683-48be-bed3-acd39b2f1f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:16.325207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f62c8-d683-48be-bed3-acd39b2f1f44 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:16.331898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:16.332904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:16.333184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:16.333460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] PATH_INFO: `/v3/users/2d5bfffc316f481183093e386a326946` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:16.334769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5bfffc316f481183093e386a326946)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:16.336619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:16.600305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5bfffc316f481183093e386a326946'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:16.679549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5bfffc316f481183093e386a326946'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:16.696676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5bfffc316f481183093e386a326946'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:16.697637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5bfffc316f481183093e386a326946'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:16.706883 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fe0db6d4-ebfc-4211-83f8-d45498ae509d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5bfffc316f481183093e386a326946'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:16.714796 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1186/2329] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:41:16 2022] DELETE /identity/v3/users/2d5bfffc316f481183093e386a326946 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:17.076918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5feb96-6997-4189-9554-7eb075c55cc0 service placement] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:17.101757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5feb96-6997-4189-9554-7eb075c55cc0 service placement] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:17.184229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:17.185039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:17.185170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:17.185450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] PATH_INFO: `/v3/projects/048c7f026f304084960d77bf46f367d8` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:17.186564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=048c7f026f304084960d77bf46f367d8)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:17.188487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:17.352301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:17.356168 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048c7f026f304084960d77bf46f367d8', 'request_id': 'req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:17.368386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048c7f026f304084960d77bf46f367d8', 'request_id': 'req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:17.463804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048c7f026f304084960d77bf46f367d8', 'request_id': 'req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:17.481937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] The token cache is being invalidate because project 048c7f026f304084960d77bf46f367d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:41:17.482223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:17.483090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f1967411-2368-4eff-9fec-d1fe7d9bcbe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:17.492409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1144/2330] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:41:17 2022] DELETE /identity/v3/projects/048c7f026f304084960d77bf46f367d8 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:17.494133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b451b7-d736-42ce-97d9-853b3abdee57 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:17.494812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b451b7-d736-42ce-97d9-853b3abdee57 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:17.494812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b451b7-d736-42ce-97d9-853b3abdee57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:17.591532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-59b451b7-d736-42ce-97d9-853b3abdee57 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:17.592850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-59b451b7-d736-42ce-97d9-853b3abdee57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:17.648904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:17.668627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1187/2331] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:17.681147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dd356-697d-498f-9746-bec67094a6ce None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:17.686856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dd356-697d-498f-9746-bec67094a6ce None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:17.686856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5dd356-697d-498f-9746-bec67094a6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:17.703974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-bf5dd356-697d-498f-9746-bec67094a6ce None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:17.706070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-bf5dd356-697d-498f-9746-bec67094a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:17.716638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1145/2332] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:17.722268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b451b7-d736-42ce-97d9-853b3abdee57 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:17.733257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b451b7-d736-42ce-97d9-853b3abdee57 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:17.739525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0171d06-15b5-446c-b4dd-b0014f0b0873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:17.740131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0171d06-15b5-446c-b4dd-b0014f0b0873 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:17.744263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0171d06-15b5-446c-b4dd-b0014f0b0873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:17.744263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0171d06-15b5-446c-b4dd-b0014f0b0873 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:17.744263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0171d06-15b5-446c-b4dd-b0014f0b0873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:41:17.744263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0171d06-15b5-446c-b4dd-b0014f0b0873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:17.744263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0171d06-15b5-446c-b4dd-b0014f0b0873 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:17.757991 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1188/2333] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:41:17.763066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dd356-697d-498f-9746-bec67094a6ce None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:17.772746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5dd356-697d-498f-9746-bec67094a6ce None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:17.777609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904d305-fa0e-4cb3-9a96-664293b88d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:17.778214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d305-fa0e-4cb3-9a96-664293b88d2b admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:17.778490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d305-fa0e-4cb3-9a96-664293b88d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:17.778757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904d305-fa0e-4cb3-9a96-664293b88d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:17.779419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904d305-fa0e-4cb3-9a96-664293b88d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:17.781500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904d305-fa0e-4cb3-9a96-664293b88d2b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:17.925022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1146/2334] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:41:17 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:41:17.926364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0171d06-15b5-446c-b4dd-b0014f0b0873 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:17.937514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0171d06-15b5-446c-b4dd-b0014f0b0873 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:18.006138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25dd68-87d8-46e3-b87e-dfbbbf3a30de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:18.006945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25dd68-87d8-46e3-b87e-dfbbbf3a30de admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:18.007329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25dd68-87d8-46e3-b87e-dfbbbf3a30de admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:18.007692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25dd68-87d8-46e3-b87e-dfbbbf3a30de admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:18.008445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25dd68-87d8-46e3-b87e-dfbbbf3a30de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:18.011832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25dd68-87d8-46e3-b87e-dfbbbf3a30de admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:18.120455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1189/2335] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:41:17 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:41:18.124976 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904d305-fa0e-4cb3-9a96-664293b88d2b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:18.140460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904d305-fa0e-4cb3-9a96-664293b88d2b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:18.146642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e645b-2ca7-48b4-9dda-61d69c5c5471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:18.147177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e645b-2ca7-48b4-9dda-61d69c5c5471 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:18.147482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e645b-2ca7-48b4-9dda-61d69c5c5471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:18.147750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e645b-2ca7-48b4-9dda-61d69c5c5471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:18.148390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e645b-2ca7-48b4-9dda-61d69c5c5471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:18.150402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e645b-2ca7-48b4-9dda-61d69c5c5471 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:18.162572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1147/2336] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:41:18.172251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25dd68-87d8-46e3-b87e-dfbbbf3a30de admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:18.182906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25dd68-87d8-46e3-b87e-dfbbbf3a30de admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:18.187684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf42d3-d6d3-4549-b3c1-fd401c82e9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:18.188280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf42d3-d6d3-4549-b3c1-fd401c82e9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:18.192671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf42d3-d6d3-4549-b3c1-fd401c82e9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:18.192671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf42d3-d6d3-4549-b3c1-fd401c82e9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:18.192671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf42d3-d6d3-4549-b3c1-fd401c82e9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:18.192671 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf42d3-d6d3-4549-b3c1-fd401c82e9e6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:18.210114 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1190/2337] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:41:18.214493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e645b-2ca7-48b4-9dda-61d69c5c5471 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:18.230007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e645b-2ca7-48b4-9dda-61d69c5c5471 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:18.234759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea671-b3dc-41c4-a26b-4750f50b7e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:18.235410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea671-b3dc-41c4-a26b-4750f50b7e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:18.236296 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea671-b3dc-41c4-a26b-4750f50b7e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:18.236727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ea671-b3dc-41c4-a26b-4750f50b7e54 admin admin] PATH_INFO: `/v3/projects/94ddc47b253841f6911e0368833b7d52/users/49baa99380cf4acba2240bbd9486b0ab/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:18.262255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ea671-b3dc-41c4-a26b-4750f50b7e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ddc47b253841f6911e0368833b7d52, user_id=49baa99380cf4acba2240bbd9486b0ab, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:18.267513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ea671-b3dc-41c4-a26b-4750f50b7e54 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:18.377452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1148/2338] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:41:18 2022] PUT /identity/v3/projects/94ddc47b253841f6911e0368833b7d52/users/49baa99380cf4acba2240bbd9486b0ab/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:18.388864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e7447-faf0-4018-918c-31cd35566864 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:18.389225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e7447-faf0-4018-918c-31cd35566864 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:18.389498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e7447-faf0-4018-918c-31cd35566864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:18.419427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8e6e7447-faf0-4018-918c-31cd35566864 None None] MFA Rules not processed for user `49baa99380cf4acba2240bbd9486b0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:18.420582 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8e6e7447-faf0-4018-918c-31cd35566864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:18.502845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1191/2339] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:18.520197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea671-b3dc-41c4-a26b-4750f50b7e54 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:18.522406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-870ea671-b3dc-41c4-a26b-4750f50b7e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:18.669528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ea671-b3dc-41c4-a26b-4750f50b7e54 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:18.677031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fce92-8a48-4162-a4fa-3e9eeb148bf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:18.677654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fce92-8a48-4162-a4fa-3e9eeb148bf3 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:18.677932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fce92-8a48-4162-a4fa-3e9eeb148bf3 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:18.678202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fce92-8a48-4162-a4fa-3e9eeb148bf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:18.687677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818fce92-8a48-4162-a4fa-3e9eeb148bf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:18.689499 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818fce92-8a48-4162-a4fa-3e9eeb148bf3 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:18.701273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1149/2340] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:41:28.498537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:28.700086 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=4,keystone:SELECT=80,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:30.757587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e7447-faf0-4018-918c-31cd35566864 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:30.823713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e7447-faf0-4018-918c-31cd35566864 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:30.828465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:30.829812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:30.830150 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:30.830538 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] PATH_INFO: `/v3/users/49baa99380cf4acba2240bbd9486b0ab` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:30.831685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] RBAC: Authorizing `identity:delete_user(user_id=49baa99380cf4acba2240bbd9486b0ab)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:30.833705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:31.036500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49baa99380cf4acba2240bbd9486b0ab'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:31.125738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49baa99380cf4acba2240bbd9486b0ab'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:31.140027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49baa99380cf4acba2240bbd9486b0ab'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:31.141046 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49baa99380cf4acba2240bbd9486b0ab'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:31.148674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49baa99380cf4acba2240bbd9486b0ab'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:31.155529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1192/2341] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:41:30 2022] DELETE /identity/v3/users/49baa99380cf4acba2240bbd9486b0ab => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:31.160281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468af3ce-19cb-4dd9-b8b3-666d17581492 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:31.160821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468af3ce-19cb-4dd9-b8b3-666d17581492 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:31.160927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468af3ce-19cb-4dd9-b8b3-666d17581492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:31.194588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-468af3ce-19cb-4dd9-b8b3-666d17581492 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:31.196024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-468af3ce-19cb-4dd9-b8b3-666d17581492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:31.266082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1150/2342] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:31.276310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:31.286725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:31.353842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fb6cb-8f57-487b-a146-ef409215842d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:31.357701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf89ef-ef47-44e4-9598-5074e96b8711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:31.358318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf89ef-ef47-44e4-9598-5074e96b8711 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:31.358595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf89ef-ef47-44e4-9598-5074e96b8711 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:31.358865 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf89ef-ef47-44e4-9598-5074e96b8711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:31.369437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf89ef-ef47-44e4-9598-5074e96b8711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:31.370869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf89ef-ef47-44e4-9598-5074e96b8711 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:31.381198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1193/2343] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:41:32.047276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468af3ce-19cb-4dd9-b8b3-666d17581492 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:32.052156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-468af3ce-19cb-4dd9-b8b3-666d17581492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:32.066001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468af3ce-19cb-4dd9-b8b3-666d17581492 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:32.071430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:32.072848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:32.072848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:32.073269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] PATH_INFO: `/v3/projects/94ddc47b253841f6911e0368833b7d52` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:32.074701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ddc47b253841f6911e0368833b7d52)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:32.076729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:32.259389 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ddc47b253841f6911e0368833b7d52', 'request_id': 'req-bdc90b0e-988c-442f-baeb-7db18266c48f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:32.273098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ddc47b253841f6911e0368833b7d52', 'request_id': 'req-bdc90b0e-988c-442f-baeb-7db18266c48f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:32.377056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ddc47b253841f6911e0368833b7d52', 'request_id': 'req-bdc90b0e-988c-442f-baeb-7db18266c48f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:32.390412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] The token cache is being invalidate because project 94ddc47b253841f6911e0368833b7d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:41:32.390835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:32.392421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bdc90b0e-988c-442f-baeb-7db18266c48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:32.394515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1151/2344] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:41:32 2022] DELETE /identity/v3/projects/94ddc47b253841f6911e0368833b7d52 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:32.400604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a788f7-ad8b-4807-adce-fa62ed678ab3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:32.401330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a788f7-ad8b-4807-adce-fa62ed678ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:32.401330 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a788f7-ad8b-4807-adce-fa62ed678ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:32.433626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f4a788f7-ad8b-4807-adce-fa62ed678ab3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:32.435726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f4a788f7-ad8b-4807-adce-fa62ed678ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:32.445380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:32.515718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1194/2345] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:32.524143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cbdac-2689-4538-9a73-c342ac04ea09 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:32.525361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cbdac-2689-4538-9a73-c342ac04ea09 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:32.525361 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264cbdac-2689-4538-9a73-c342ac04ea09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:32.547375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-264cbdac-2689-4538-9a73-c342ac04ea09 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:32.549511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-264cbdac-2689-4538-9a73-c342ac04ea09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:32.559776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1152/2346] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:41:32.565107 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a788f7-ad8b-4807-adce-fa62ed678ab3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:32.575371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a788f7-ad8b-4807-adce-fa62ed678ab3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:32.580304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ab7df-2dac-4153-bd0b-cb4d5b885452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:32.585241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ab7df-2dac-4153-bd0b-cb4d5b885452 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:32.585241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ab7df-2dac-4153-bd0b-cb4d5b885452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:32.585241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ab7df-2dac-4153-bd0b-cb4d5b885452 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:32.585241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ab7df-2dac-4153-bd0b-cb4d5b885452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:41:32.585241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ab7df-2dac-4153-bd0b-cb4d5b885452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:32.585241 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ab7df-2dac-4153-bd0b-cb4d5b885452 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:32.601305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1195/2347] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:41:32.602454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cbdac-2689-4538-9a73-c342ac04ea09 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:32.613612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264cbdac-2689-4538-9a73-c342ac04ea09 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:32.618895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c6d48-fdc1-4e08-a1e3-4efa4ac5bf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:32.619508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c6d48-fdc1-4e08-a1e3-4efa4ac5bf93 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:32.619788 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c6d48-fdc1-4e08-a1e3-4efa4ac5bf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:32.620067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c6d48-fdc1-4e08-a1e3-4efa4ac5bf93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:32.620698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c6d48-fdc1-4e08-a1e3-4efa4ac5bf93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:32.625339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c6d48-fdc1-4e08-a1e3-4efa4ac5bf93 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:32.763642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1153/2348] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:41:32 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:41:32.765111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ab7df-2dac-4153-bd0b-cb4d5b885452 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:32.776323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ab7df-2dac-4153-bd0b-cb4d5b885452 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:32.841725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341441a-7f4a-4ee6-9a8f-845d91feed35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:32.842322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341441a-7f4a-4ee6-9a8f-845d91feed35 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:32.842595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341441a-7f4a-4ee6-9a8f-845d91feed35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:32.842862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341441a-7f4a-4ee6-9a8f-845d91feed35 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:32.843529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341441a-7f4a-4ee6-9a8f-845d91feed35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:32.845520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341441a-7f4a-4ee6-9a8f-845d91feed35 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:33.007481 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1196/2349] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:41:32 2022] POST /identity/v3/users => generated 284 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:41:33.008674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c6d48-fdc1-4e08-a1e3-4efa4ac5bf93 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:33.020165 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c6d48-fdc1-4e08-a1e3-4efa4ac5bf93 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:33.026040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f509e-4cd0-493f-97bf-3e5a1174433e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:33.026661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f509e-4cd0-493f-97bf-3e5a1174433e admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:33.027194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f509e-4cd0-493f-97bf-3e5a1174433e admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:33.027194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f509e-4cd0-493f-97bf-3e5a1174433e admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:33.027824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f509e-4cd0-493f-97bf-3e5a1174433e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:33.029725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f509e-4cd0-493f-97bf-3e5a1174433e admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:33.054977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1154/2350] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:41:33.055216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341441a-7f4a-4ee6-9a8f-845d91feed35 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:33.060477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341441a-7f4a-4ee6-9a8f-845d91feed35 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:33.066852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787dfd9-a031-4d31-846a-4a8020e1336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:33.067490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787dfd9-a031-4d31-846a-4a8020e1336a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:33.074780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787dfd9-a031-4d31-846a-4a8020e1336a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:33.074780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787dfd9-a031-4d31-846a-4a8020e1336a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:33.074780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787dfd9-a031-4d31-846a-4a8020e1336a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:33.074780 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787dfd9-a031-4d31-846a-4a8020e1336a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:33.082519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1197/2351] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:41:33.086862 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f509e-4cd0-493f-97bf-3e5a1174433e admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:33.098064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f509e-4cd0-493f-97bf-3e5a1174433e admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:33.105106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42dd50-76a2-449b-be32-aad850f58d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:33.106133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42dd50-76a2-449b-be32-aad850f58d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:33.106633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42dd50-76a2-449b-be32-aad850f58d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:33.106916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42dd50-76a2-449b-be32-aad850f58d14 admin admin] PATH_INFO: `/v3/projects/ed4c03d9b68b4338a731577ddcda0382/users/61a069438e7547d8a5cd7d0ebfa21e7c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:33.127303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42dd50-76a2-449b-be32-aad850f58d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4c03d9b68b4338a731577ddcda0382, user_id=61a069438e7547d8a5cd7d0ebfa21e7c, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:33.130161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42dd50-76a2-449b-be32-aad850f58d14 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:33.276420 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1155/2352] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:41:33 2022] PUT /identity/v3/projects/ed4c03d9b68b4338a731577ddcda0382/users/61a069438e7547d8a5cd7d0ebfa21e7c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:41:33.289479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c1171-c714-4970-84f4-8c30b1c16be5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:33.289762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c1171-c714-4970-84f4-8c30b1c16be5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:33.290157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c1171-c714-4970-84f4-8c30b1c16be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:33.319930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a98c1171-c714-4970-84f4-8c30b1c16be5 None None] MFA Rules not processed for user `61a069438e7547d8a5cd7d0ebfa21e7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:33.321217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a98c1171-c714-4970-84f4-8c30b1c16be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:33.394828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1198/2353] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:41:33.399301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42dd50-76a2-449b-be32-aad850f58d14 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:33.412480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:33.470522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42dd50-76a2-449b-be32-aad850f58d14 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:33.476743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54a6f1-e513-41c1-bdab-5158aa973f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:33.477400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54a6f1-e513-41c1-bdab-5158aa973f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:33.477683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54a6f1-e513-41c1-bdab-5158aa973f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:33.478019 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54a6f1-e513-41c1-bdab-5158aa973f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:33.478595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54a6f1-e513-41c1-bdab-5158aa973f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:33.480534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54a6f1-e513-41c1-bdab-5158aa973f7d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:33.631010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1156/2354] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:41:33 2022] POST /identity/v3/projects => generated 367 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:41:33.635490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c1171-c714-4970-84f4-8c30b1c16be5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:33.649067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c1171-c714-4970-84f4-8c30b1c16be5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:33.654677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:33.711845 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594498b-9691-4e29-ade8-36ef35655ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:33.712472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594498b-9691-4e29-ade8-36ef35655ecc admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:33.713627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594498b-9691-4e29-ade8-36ef35655ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:33.713627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594498b-9691-4e29-ade8-36ef35655ecc admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:33.713948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594498b-9691-4e29-ade8-36ef35655ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:33.715643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594498b-9691-4e29-ade8-36ef35655ecc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:33.839907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1199/2355] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:41:33 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:41:33.844990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54a6f1-e513-41c1-bdab-5158aa973f7d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:33.855989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54a6f1-e513-41c1-bdab-5158aa973f7d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:33.861568 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0db2ea-267c-420a-9f42-5c9f126b10a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:33.862183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0db2ea-267c-420a-9f42-5c9f126b10a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:33.862473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0db2ea-267c-420a-9f42-5c9f126b10a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:33.862723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0db2ea-267c-420a-9f42-5c9f126b10a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:33.863353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0db2ea-267c-420a-9f42-5c9f126b10a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:33.865056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0db2ea-267c-420a-9f42-5c9f126b10a3 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:33.876562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1157/2356] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:41:33.881115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594498b-9691-4e29-ade8-36ef35655ecc admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:33.893142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594498b-9691-4e29-ade8-36ef35655ecc admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:33.898322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98295344-446c-4656-9718-7aede1125294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:33.898987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98295344-446c-4656-9718-7aede1125294 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:33.899267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98295344-446c-4656-9718-7aede1125294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:33.899543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98295344-446c-4656-9718-7aede1125294 admin admin] PATH_INFO: `/v3/projects/0231ed5ae5934cb292add35631845ea0/users/abf383df25584820bf6a54c421f58925/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:33.915957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98295344-446c-4656-9718-7aede1125294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0231ed5ae5934cb292add35631845ea0, user_id=abf383df25584820bf6a54c421f58925, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:33.918721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98295344-446c-4656-9718-7aede1125294 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:34.028483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1200/2357] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:41:33 2022] PUT /identity/v3/projects/0231ed5ae5934cb292add35631845ea0/users/abf383df25584820bf6a54c421f58925/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:34.035070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0db2ea-267c-420a-9f42-5c9f126b10a3 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:34.048858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0db2ea-267c-420a-9f42-5c9f126b10a3 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:34.127669 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47676c6a-b277-4607-b178-d71ed1de8aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:34.128276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47676c6a-b277-4607-b178-d71ed1de8aae admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:34.128550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47676c6a-b277-4607-b178-d71ed1de8aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:34.128843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47676c6a-b277-4607-b178-d71ed1de8aae admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:34.129476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47676c6a-b277-4607-b178-d71ed1de8aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:34.131066 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47676c6a-b277-4607-b178-d71ed1de8aae admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:34.142829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1158/2358] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:41:34.146296 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98295344-446c-4656-9718-7aede1125294 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:34.159106 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98295344-446c-4656-9718-7aede1125294 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:34.163758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddff11-428b-49f2-9213-10c36b48a027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:34.164415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddff11-428b-49f2-9213-10c36b48a027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:34.164687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddff11-428b-49f2-9213-10c36b48a027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:34.165052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddff11-428b-49f2-9213-10c36b48a027 admin admin] PATH_INFO: `/v3/projects/0231ed5ae5934cb292add35631845ea0/users/abf383df25584820bf6a54c421f58925/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:34.167048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddff11-428b-49f2-9213-10c36b48a027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0231ed5ae5934cb292add35631845ea0, user_id=abf383df25584820bf6a54c421f58925, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:34.169771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddff11-428b-49f2-9213-10c36b48a027 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:34.181861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1201/2359] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:41:34 2022] PUT /identity/v3/projects/0231ed5ae5934cb292add35631845ea0/users/abf383df25584820bf6a54c421f58925/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:41:34.192921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b0d36-71bd-4edc-86bf-0f4c9e594344 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:34.193274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b0d36-71bd-4edc-86bf-0f4c9e594344 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:34.193551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b0d36-71bd-4edc-86bf-0f4c9e594344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:34.222395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-668b0d36-71bd-4edc-86bf-0f4c9e594344 None None] MFA Rules not processed for user `abf383df25584820bf6a54c421f58925`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:34.223586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-668b0d36-71bd-4edc-86bf-0f4c9e594344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:34.316031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1159/2360] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:34.326898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddff11-428b-49f2-9213-10c36b48a027 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:34.329278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-67ddff11-428b-49f2-9213-10c36b48a027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:34.416901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddff11-428b-49f2-9213-10c36b48a027 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:34.420568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e891466-609d-4dbd-941b-f068a82e41dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:34.421211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e891466-609d-4dbd-941b-f068a82e41dc service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:34.421497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e891466-609d-4dbd-941b-f068a82e41dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:34.421761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e891466-609d-4dbd-941b-f068a82e41dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:34.432024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e891466-609d-4dbd-941b-f068a82e41dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:34.433491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e891466-609d-4dbd-941b-f068a82e41dc service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:34.444187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1202/2361] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:41:34.698133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b0d36-71bd-4edc-86bf-0f4c9e594344 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:34.699628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-668b0d36-71bd-4edc-86bf-0f4c9e594344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:34.715172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b0d36-71bd-4edc-86bf-0f4c9e594344 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:34.719090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebdca8-d0cf-4be2-ab01-02091c0aa4fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:34.720483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebdca8-d0cf-4be2-ab01-02091c0aa4fe service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:34.720765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebdca8-d0cf-4be2-ab01-02091c0aa4fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:34.721073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebdca8-d0cf-4be2-ab01-02091c0aa4fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:34.725598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:34.786063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebdca8-d0cf-4be2-ab01-02091c0aa4fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:34.787675 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eebdca8-d0cf-4be2-ab01-02091c0aa4fe service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:34.796918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1160/2362] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:41:38.209556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d20f-4673-4c21-a0d0-08e6d948b6be None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:38.210040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d20f-4673-4c21-a0d0-08e6d948b6be None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:38.210040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d20f-4673-4c21-a0d0-08e6d948b6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:38.260789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e9c8d20f-4673-4c21-a0d0-08e6d948b6be None None] MFA Rules not processed for user `0031939e16194f63b99e6f27f700fa8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:38.261846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e9c8d20f-4673-4c21-a0d0-08e6d948b6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:38.280877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:38.361513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1203/2363] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:40.660529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebdca8-d0cf-4be2-ab01-02091c0aa4fe service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:40.731612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebdca8-d0cf-4be2-ab01-02091c0aa4fe service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:40.737025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:40.738269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:40.738269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:40.738489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] PATH_INFO: `/v3/users/61a069438e7547d8a5cd7d0ebfa21e7c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:40.739471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a069438e7547d8a5cd7d0ebfa21e7c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:40.741536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:40.992719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a069438e7547d8a5cd7d0ebfa21e7c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:40.993344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a069438e7547d8a5cd7d0ebfa21e7c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:40.993963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a069438e7547d8a5cd7d0ebfa21e7c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:41.002271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a069438e7547d8a5cd7d0ebfa21e7c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:41.016055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a069438e7547d8a5cd7d0ebfa21e7c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:41.119721 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1161/2364] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:41:40 2022] DELETE /identity/v3/users/61a069438e7547d8a5cd7d0ebfa21e7c => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:41.124658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bb312-0914-4803-b239-e068a8b10d0e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:41.125256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bb312-0914-4803-b239-e068a8b10d0e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:41.125256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bb312-0914-4803-b239-e068a8b10d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:41.146250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-270bb312-0914-4803-b239-e068a8b10d0e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:41.147480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-270bb312-0914-4803-b239-e068a8b10d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:41.217973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1204/2365] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:41.227818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:41.229244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:41.258465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bb312-0914-4803-b239-e068a8b10d0e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:41.259772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-270bb312-0914-4803-b239-e068a8b10d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:41.260925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:41.295219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ead48e-7c19-4339-b2c6-6c2a5db12e44 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:41.302364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71751731-6854-468b-9852-af968f076871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:41.303292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71751731-6854-468b-9852-af968f076871 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:41.303573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71751731-6854-468b-9852-af968f076871 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:41.303843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71751731-6854-468b-9852-af968f076871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:41.320868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71751731-6854-468b-9852-af968f076871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:41.324006 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71751731-6854-468b-9852-af968f076871 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:41.338289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1162/2366] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:41:41.366988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bb312-0914-4803-b239-e068a8b10d0e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:41.373757 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b835f4-ffeb-40ed-8bb6-faa6cc04ae58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:41.374484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b835f4-ffeb-40ed-8bb6-faa6cc04ae58 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:41.375019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b835f4-ffeb-40ed-8bb6-faa6cc04ae58 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:41.375239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b835f4-ffeb-40ed-8bb6-faa6cc04ae58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:41.377805 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-18b835f4-ffeb-40ed-8bb6-faa6cc04ae58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:41.448720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b835f4-ffeb-40ed-8bb6-faa6cc04ae58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:41.450551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b835f4-ffeb-40ed-8bb6-faa6cc04ae58 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:41.463689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1205/2367] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:41:42.054370 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71751731-6854-468b-9852-af968f076871 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:42.055571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-71751731-6854-468b-9852-af968f076871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:42.083730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71751731-6854-468b-9852-af968f076871 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:42.097867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:42.098586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:42.098887 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:42.099188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] PATH_INFO: `/v3/projects/ed4c03d9b68b4338a731577ddcda0382` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:42.100495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4c03d9b68b4338a731577ddcda0382)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:42.103045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:42.327332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4c03d9b68b4338a731577ddcda0382', 'request_id': 'req-8895ca4c-5155-4987-a810-4cd967c78a91'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:42.335404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4c03d9b68b4338a731577ddcda0382', 'request_id': 'req-8895ca4c-5155-4987-a810-4cd967c78a91'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:42.508571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4c03d9b68b4338a731577ddcda0382', 'request_id': 'req-8895ca4c-5155-4987-a810-4cd967c78a91'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:42.529352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] The token cache is being invalidate because project ed4c03d9b68b4338a731577ddcda0382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:41:42.529352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:42.529352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:42.529352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1163/2368] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:41:42 2022] DELETE /identity/v3/projects/ed4c03d9b68b4338a731577ddcda0382 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:42.534477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b835f4-ffeb-40ed-8bb6-faa6cc04ae58 service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:42.541250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:42.549413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b835f4-ffeb-40ed-8bb6-faa6cc04ae58 service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:42.657496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:42.658529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:42.659083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:42.659529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] PATH_INFO: `/v3/users/abf383df25584820bf6a54c421f58925` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:42.661299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf383df25584820bf6a54c421f58925)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:42.663606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:42.880712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf383df25584820bf6a54c421f58925'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:42.976926 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf383df25584820bf6a54c421f58925'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:42.996188 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf383df25584820bf6a54c421f58925'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:42.998021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf383df25584820bf6a54c421f58925'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:43.013148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b6a21576-457e-4c0f-9434-51e89b049a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf383df25584820bf6a54c421f58925'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:43.028252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1206/2369] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:41:42 2022] DELETE /identity/v3/users/abf383df25584820bf6a54c421f58925 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:41:43.782244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:43.784605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:43.805442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895ca4c-5155-4987-a810-4cd967c78a91 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:43.959474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:43.960226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:43.960582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:43.960961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] PATH_INFO: `/v3/projects/0231ed5ae5934cb292add35631845ea0` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:43.962354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0231ed5ae5934cb292add35631845ea0)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:43.965065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:44.212608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0231ed5ae5934cb292add35631845ea0', 'request_id': 'req-16f28306-65bd-4904-b63a-2c6e28aa484c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:44.217080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:44.226622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0231ed5ae5934cb292add35631845ea0', 'request_id': 'req-16f28306-65bd-4904-b63a-2c6e28aa484c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:44.360698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0231ed5ae5934cb292add35631845ea0', 'request_id': 'req-16f28306-65bd-4904-b63a-2c6e28aa484c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:44.383167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] The token cache is being invalidate because project 0231ed5ae5934cb292add35631845ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:41:44.383789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:44.384953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-16f28306-65bd-4904-b63a-2c6e28aa484c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:44.388332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1164/2370] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:41:43 2022] DELETE /identity/v3/projects/0231ed5ae5934cb292add35631845ea0 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:44.398398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af18141-df5f-4c83-b3ef-3efd230078d5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:44.398398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af18141-df5f-4c83-b3ef-3efd230078d5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:44.398398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af18141-df5f-4c83-b3ef-3efd230078d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:44.419121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2af18141-df5f-4c83-b3ef-3efd230078d5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:44.421262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2af18141-df5f-4c83-b3ef-3efd230078d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:44.489813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1207/2371] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:44.498312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e178d-f02f-4dbe-bc4d-2187ea4e9589 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:44.498637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e178d-f02f-4dbe-bc4d-2187ea4e9589 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:44.498825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e178d-f02f-4dbe-bc4d-2187ea4e9589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:44.519613 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c69e178d-f02f-4dbe-bc4d-2187ea4e9589 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:44.520868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c69e178d-f02f-4dbe-bc4d-2187ea4e9589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:44.529253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1165/2372] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:44.534366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af18141-df5f-4c83-b3ef-3efd230078d5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:44.544343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af18141-df5f-4c83-b3ef-3efd230078d5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:44.549474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae1673-ab4a-46f7-b210-d723f5db350d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:44.550220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae1673-ab4a-46f7-b210-d723f5db350d admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:44.550561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae1673-ab4a-46f7-b210-d723f5db350d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:44.550901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae1673-ab4a-46f7-b210-d723f5db350d admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:44.551721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae1673-ab4a-46f7-b210-d723f5db350d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:41:44.552093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae1673-ab4a-46f7-b210-d723f5db350d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:44.553778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae1673-ab4a-46f7-b210-d723f5db350d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:44.568521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1208/2373] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:41:44.574440 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e178d-f02f-4dbe-bc4d-2187ea4e9589 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:44.586452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e178d-f02f-4dbe-bc4d-2187ea4e9589 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:44.592932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83767ea8-431f-44fa-a4e0-53dd3b586368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:44.593561 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83767ea8-431f-44fa-a4e0-53dd3b586368 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:44.593837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83767ea8-431f-44fa-a4e0-53dd3b586368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:44.595337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83767ea8-431f-44fa-a4e0-53dd3b586368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:44.596050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83767ea8-431f-44fa-a4e0-53dd3b586368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:44.598633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83767ea8-431f-44fa-a4e0-53dd3b586368 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:44.724926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1166/2374] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:41:44 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:41:44.726472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae1673-ab4a-46f7-b210-d723f5db350d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:44.753819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae1673-ab4a-46f7-b210-d723f5db350d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:44.764226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:44.846368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb8472-dd2f-4ae0-a30c-f558d63edea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:44.846368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb8472-dd2f-4ae0-a30c-f558d63edea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:44.846368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb8472-dd2f-4ae0-a30c-f558d63edea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:44.846368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb8472-dd2f-4ae0-a30c-f558d63edea1 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:44.846726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb8472-dd2f-4ae0-a30c-f558d63edea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:44.846726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb8472-dd2f-4ae0-a30c-f558d63edea1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:44.965486 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1209/2375] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:41:44 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:41:44.969884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83767ea8-431f-44fa-a4e0-53dd3b586368 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:44.982633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83767ea8-431f-44fa-a4e0-53dd3b586368 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:44.992693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93fafa-c748-44e5-8b75-6f697428586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:44.993767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93fafa-c748-44e5-8b75-6f697428586f admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:44.994258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93fafa-c748-44e5-8b75-6f697428586f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:44.994900 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93fafa-c748-44e5-8b75-6f697428586f admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:44.996073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93fafa-c748-44e5-8b75-6f697428586f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:44.999664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93fafa-c748-44e5-8b75-6f697428586f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:45.019864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1167/2376] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:41:45.023063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb8472-dd2f-4ae0-a30c-f558d63edea1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:45.040583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb8472-dd2f-4ae0-a30c-f558d63edea1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:45.049198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe846bdf-9cfc-45c8-9d41-bd403703f8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:45.049812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe846bdf-9cfc-45c8-9d41-bd403703f8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:45.050093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe846bdf-9cfc-45c8-9d41-bd403703f8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:45.050364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe846bdf-9cfc-45c8-9d41-bd403703f8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:45.051078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe846bdf-9cfc-45c8-9d41-bd403703f8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:45.052660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe846bdf-9cfc-45c8-9d41-bd403703f8f4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:45.065434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1210/2377] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:41:45.069661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93fafa-c748-44e5-8b75-6f697428586f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:45.084187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93fafa-c748-44e5-8b75-6f697428586f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:45.094106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beef121-1f78-4f14-b8f5-8629434b968f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:45.096370 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beef121-1f78-4f14-b8f5-8629434b968f admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:45.096370 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beef121-1f78-4f14-b8f5-8629434b968f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:45.096901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beef121-1f78-4f14-b8f5-8629434b968f admin admin] PATH_INFO: `/v3/projects/f083bdcc6afb414d9553e18acbd41890/users/7e393d03ea064979978fb56c38aeaf03/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:45.118284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beef121-1f78-4f14-b8f5-8629434b968f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f083bdcc6afb414d9553e18acbd41890, user_id=7e393d03ea064979978fb56c38aeaf03, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:45.121008 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beef121-1f78-4f14-b8f5-8629434b968f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:45.222221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1168/2378] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:41:45 2022] PUT /identity/v3/projects/f083bdcc6afb414d9553e18acbd41890/users/7e393d03ea064979978fb56c38aeaf03/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:45.234374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e541da0-c8e6-4e2d-9e83-ebccf90e7e18 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:45.234727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e541da0-c8e6-4e2d-9e83-ebccf90e7e18 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:45.235597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e541da0-c8e6-4e2d-9e83-ebccf90e7e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:45.267987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6e541da0-c8e6-4e2d-9e83-ebccf90e7e18 None None] MFA Rules not processed for user `7e393d03ea064979978fb56c38aeaf03`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:45.269892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6e541da0-c8e6-4e2d-9e83-ebccf90e7e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:45.350911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1211/2379] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:45.356645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beef121-1f78-4f14-b8f5-8629434b968f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:45.361350 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6beef121-1f78-4f14-b8f5-8629434b968f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:45.363331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e541da0-c8e6-4e2d-9e83-ebccf90e7e18 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:45.364908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6e541da0-c8e6-4e2d-9e83-ebccf90e7e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:45.466470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e541da0-c8e6-4e2d-9e83-ebccf90e7e18 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:45.473085 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c9ac4-4f69-4588-86c7-84cc25db160c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:45.474012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c9ac4-4f69-4588-86c7-84cc25db160c service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:45.474453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c9ac4-4f69-4588-86c7-84cc25db160c service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:45.474882 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c9ac4-4f69-4588-86c7-84cc25db160c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:45.491512 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c9ac4-4f69-4588-86c7-84cc25db160c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:45.493179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c9ac4-4f69-4588-86c7-84cc25db160c service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:45.502309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beef121-1f78-4f14-b8f5-8629434b968f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:45.506592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1212/2380] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:41:45.515967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5016a9-570f-4772-b498-c25aedf853ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:45.515967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5016a9-570f-4772-b498-c25aedf853ee service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:45.515967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5016a9-570f-4772-b498-c25aedf853ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:45.515967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5016a9-570f-4772-b498-c25aedf853ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:45.517546 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-bb5016a9-570f-4772-b498-c25aedf853ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:45.618347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5016a9-570f-4772-b498-c25aedf853ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:45.621879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5016a9-570f-4772-b498-c25aedf853ee service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:45.644104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1169/2381] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:41:49.979611 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c9ac4-4f69-4588-86c7-84cc25db160c service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:50.009446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:50.052687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c9ac4-4f69-4588-86c7-84cc25db160c service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:50.057597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:50.061891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:50.061891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:50.061891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] PATH_INFO: `/v3/users/7e393d03ea064979978fb56c38aeaf03` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:50.061891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e393d03ea064979978fb56c38aeaf03)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:50.062506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:50.351401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e393d03ea064979978fb56c38aeaf03'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:50.497520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e393d03ea064979978fb56c38aeaf03'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:50.506971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e393d03ea064979978fb56c38aeaf03'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:50.507915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e393d03ea064979978fb56c38aeaf03'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:50.515740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e393d03ea064979978fb56c38aeaf03'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:50.523362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1213/2382] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:41:49 2022] DELETE /identity/v3/users/7e393d03ea064979978fb56c38aeaf03 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:50.528670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74ee86-1f4d-400a-bdb6-cd754ad0172a None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:50.528670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74ee86-1f4d-400a-bdb6-cd754ad0172a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:50.539612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74ee86-1f4d-400a-bdb6-cd754ad0172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:50.550979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-8a74ee86-1f4d-400a-bdb6-cd754ad0172a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:50.553077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8a74ee86-1f4d-400a-bdb6-cd754ad0172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:50.568996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:50.627742 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1170/2383] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:50.642509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:50.644086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:50.711503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4774e-5be3-441f-80c1-ca4594ef2a87 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:50.715577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee4961-c136-4893-bf7d-bfe979a339d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:50.716192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee4961-c136-4893-bf7d-bfe979a339d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:50.716472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee4961-c136-4893-bf7d-bfe979a339d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:50.716744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee4961-c136-4893-bf7d-bfe979a339d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:50.731634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee4961-c136-4893-bf7d-bfe979a339d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:50.731634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee4961-c136-4893-bf7d-bfe979a339d9 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:50.742100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1214/2384] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:41:51.275220 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74ee86-1f4d-400a-bdb6-cd754ad0172a None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:51.277656 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8a74ee86-1f4d-400a-bdb6-cd754ad0172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:51.293519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74ee86-1f4d-400a-bdb6-cd754ad0172a None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:51.299716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:51.300487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:51.300874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:51.301222 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] PATH_INFO: `/v3/projects/f083bdcc6afb414d9553e18acbd41890` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:51.302503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f083bdcc6afb414d9553e18acbd41890)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:51.304650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:51.476885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f083bdcc6afb414d9553e18acbd41890', 'request_id': 'req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:51.483323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f083bdcc6afb414d9553e18acbd41890', 'request_id': 'req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:51.616244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f083bdcc6afb414d9553e18acbd41890', 'request_id': 'req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:51.628868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] The token cache is being invalidate because project f083bdcc6afb414d9553e18acbd41890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:41:51.629157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:51.629816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4c5e9d89-d631-47d8-9de8-bb89d1e15a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:41:51.631337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1171/2385] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:41:51 2022] DELETE /identity/v3/projects/f083bdcc6afb414d9553e18acbd41890 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:41:51.637726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb07714-492d-462d-bec2-278f6a25473e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:51.638244 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb07714-492d-462d-bec2-278f6a25473e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:51.638445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb07714-492d-462d-bec2-278f6a25473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:51.660866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-1cb07714-492d-462d-bec2-278f6a25473e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:51.662103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1cb07714-492d-462d-bec2-278f6a25473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:51.706443 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:51.726653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1215/2386] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:51.734477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8331-da5e-4167-97b3-8985f9cd0148 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:51.736409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8331-da5e-4167-97b3-8985f9cd0148 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:51.736409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a8331-da5e-4167-97b3-8985f9cd0148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:51.745623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb07714-492d-462d-bec2-278f6a25473e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:51.747103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1cb07714-492d-462d-bec2-278f6a25473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:51.759316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-eb9a8331-da5e-4167-97b3-8985f9cd0148 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:51.760579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-eb9a8331-da5e-4167-97b3-8985f9cd0148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:51.772939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1172/2387] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:41:51.777939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8331-da5e-4167-97b3-8985f9cd0148 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:51.788273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a8331-da5e-4167-97b3-8985f9cd0148 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:51.794522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5d28f-e4aa-40f8-b402-d91527b4fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:51.795195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5d28f-e4aa-40f8-b402-d91527b4fe13 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:51.795522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5d28f-e4aa-40f8-b402-d91527b4fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:51.795741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5d28f-e4aa-40f8-b402-d91527b4fe13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:51.796407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5d28f-e4aa-40f8-b402-d91527b4fe13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:41:51.796711 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5d28f-e4aa-40f8-b402-d91527b4fe13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:51.799418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5d28f-e4aa-40f8-b402-d91527b4fe13 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:51.816503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1173/2388] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:41:51.818499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb07714-492d-462d-bec2-278f6a25473e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:51.818694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5d28f-e4aa-40f8-b402-d91527b4fe13 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:51.823472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f8e6-fab5-445b-adcc-01e0723bd4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:51.829169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f8e6-fab5-445b-adcc-01e0723bd4df service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:51.829169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f8e6-fab5-445b-adcc-01e0723bd4df service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:51.829169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397f8e6-fab5-445b-adcc-01e0723bd4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:51.829169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3397f8e6-fab5-445b-adcc-01e0723bd4df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:51.830381 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5d28f-e4aa-40f8-b402-d91527b4fe13 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:51.835512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56678ee7-3ac9-47f6-9545-4a5bf42438e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:51.836124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56678ee7-3ac9-47f6-9545-4a5bf42438e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:51.836538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56678ee7-3ac9-47f6-9545-4a5bf42438e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:51.838587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56678ee7-3ac9-47f6-9545-4a5bf42438e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:51.838587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56678ee7-3ac9-47f6-9545-4a5bf42438e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:51.839836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56678ee7-3ac9-47f6-9545-4a5bf42438e9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:51.909185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397f8e6-fab5-445b-adcc-01e0723bd4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:51.910631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397f8e6-fab5-445b-adcc-01e0723bd4df service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:51.922750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1216/2389] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:41:51.958669 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1174/2390] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:41:51 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:41:51.960599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f8e6-fab5-445b-adcc-01e0723bd4df service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:51.972248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397f8e6-fab5-445b-adcc-01e0723bd4df service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:52.035569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb02b6-4b8c-4493-afe7-b2115d641dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:52.036250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb02b6-4b8c-4493-afe7-b2115d641dee admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:52.036529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb02b6-4b8c-4493-afe7-b2115d641dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:52.036943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb02b6-4b8c-4493-afe7-b2115d641dee admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:52.037636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb02b6-4b8c-4493-afe7-b2115d641dee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:52.039672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb02b6-4b8c-4493-afe7-b2115d641dee admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:52.161225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1217/2391] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:41:51 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:41:52.165551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56678ee7-3ac9-47f6-9545-4a5bf42438e9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:52.177032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56678ee7-3ac9-47f6-9545-4a5bf42438e9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:52.181646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dada8e8-bd66-449c-af57-e5427cc0453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:52.182238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dada8e8-bd66-449c-af57-e5427cc0453a admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:52.182514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dada8e8-bd66-449c-af57-e5427cc0453a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:52.182779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dada8e8-bd66-449c-af57-e5427cc0453a admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:52.183439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dada8e8-bd66-449c-af57-e5427cc0453a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:52.185041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dada8e8-bd66-449c-af57-e5427cc0453a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:52.195767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1175/2392] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:41:52.209180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb02b6-4b8c-4493-afe7-b2115d641dee admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:52.219647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb02b6-4b8c-4493-afe7-b2115d641dee admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:52.224709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925055c-3550-467b-bf04-f7d462f474a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:52.225337 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925055c-3550-467b-bf04-f7d462f474a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:52.225615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925055c-3550-467b-bf04-f7d462f474a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:52.225887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925055c-3550-467b-bf04-f7d462f474a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:52.226522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925055c-3550-467b-bf04-f7d462f474a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:52.228148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925055c-3550-467b-bf04-f7d462f474a6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:52.238957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1218/2393] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:41:52.243544 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dada8e8-bd66-449c-af57-e5427cc0453a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:52.255106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dada8e8-bd66-449c-af57-e5427cc0453a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:52.259726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538e416-10f3-4b79-b978-541343cea38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:52.260386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538e416-10f3-4b79-b978-541343cea38b admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:52.260659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538e416-10f3-4b79-b978-541343cea38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:52.260962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538e416-10f3-4b79-b978-541343cea38b admin admin] PATH_INFO: `/v3/projects/7b6e994f692c4732a989fadcd5afe3a8/users/d41b859907fe447888f0e5d9bc0c3b3d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:52.277242 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538e416-10f3-4b79-b978-541343cea38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6e994f692c4732a989fadcd5afe3a8, user_id=d41b859907fe447888f0e5d9bc0c3b3d, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:52.279892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538e416-10f3-4b79-b978-541343cea38b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:52.458401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1176/2394] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:41:52 2022] PUT /identity/v3/projects/7b6e994f692c4732a989fadcd5afe3a8/users/d41b859907fe447888f0e5d9bc0c3b3d/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:41:52.468749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69e6ad-53ea-4037-b978-1afa7a4f1bdc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:52.469115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69e6ad-53ea-4037-b978-1afa7a4f1bdc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:52.470109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69e6ad-53ea-4037-b978-1afa7a4f1bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:52.479990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:52.502380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4c69e6ad-53ea-4037-b978-1afa7a4f1bdc None None] MFA Rules not processed for user `d41b859907fe447888f0e5d9bc0c3b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:41:52.503593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4c69e6ad-53ea-4037-b978-1afa7a4f1bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:52.585387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1219/2395] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:41:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:41:52.597632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538e416-10f3-4b79-b978-541343cea38b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:41:52.599773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f538e416-10f3-4b79-b978-541343cea38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:41:52.642345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:41:52.680942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538e416-10f3-4b79-b978-541343cea38b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:41:52.684743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168405b2-9eb5-4cf8-9414-8069d7b38a39 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ac781cf2d84d95a789724e74e12acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:41:52.688961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168405b2-9eb5-4cf8-9414-8069d7b38a39 service glance] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:41:52.688961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168405b2-9eb5-4cf8-9414-8069d7b38a39 service glance] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:41:52.688961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168405b2-9eb5-4cf8-9414-8069d7b38a39 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:41:52.696896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168405b2-9eb5-4cf8-9414-8069d7b38a39 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:41:52.698910 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168405b2-9eb5-4cf8-9414-8069d7b38a39 service glance] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:41:52.712234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1177/2396] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:41:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:02.675011 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=30 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:03.090576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=15 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:39.815013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69e6ad-53ea-4037-b978-1afa7a4f1bdc None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:39.880705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69e6ad-53ea-4037-b978-1afa7a4f1bdc None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:39.886446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:39.887120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:39.887451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:39.887872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] PATH_INFO: `/v3/users/d41b859907fe447888f0e5d9bc0c3b3d` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:39.889146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41b859907fe447888f0e5d9bc0c3b3d)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:39.891384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:40.169279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41b859907fe447888f0e5d9bc0c3b3d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:40.300358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41b859907fe447888f0e5d9bc0c3b3d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:40.314284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41b859907fe447888f0e5d9bc0c3b3d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:40.314683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41b859907fe447888f0e5d9bc0c3b3d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:40.321690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41b859907fe447888f0e5d9bc0c3b3d'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:40.327850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1220/2397] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:42:39 2022] DELETE /identity/v3/users/d41b859907fe447888f0e5d9bc0c3b3d => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:40.332533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abea41-fd88-416e-ae2c-9eb488bb6cd5 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:40.333062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abea41-fd88-416e-ae2c-9eb488bb6cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:40.333162 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abea41-fd88-416e-ae2c-9eb488bb6cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:40.354577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d4abea41-fd88-416e-ae2c-9eb488bb6cd5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:40.355771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d4abea41-fd88-416e-ae2c-9eb488bb6cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:40.418307 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1178/2398] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:40.428240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:40.429688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:40.501690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a79960-e82d-4664-b15a-7db0e3c524f6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:40.505578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f0be9-9a72-4a20-b605-9b0791daeb37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:40.506184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f0be9-9a72-4a20-b605-9b0791daeb37 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:40.506462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f0be9-9a72-4a20-b605-9b0791daeb37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:40.506731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f0be9-9a72-4a20-b605-9b0791daeb37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:40.516402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f0be9-9a72-4a20-b605-9b0791daeb37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:40.517887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f0be9-9a72-4a20-b605-9b0791daeb37 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:40.527735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1221/2399] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:41.006980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abea41-fd88-416e-ae2c-9eb488bb6cd5 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:41.008565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d4abea41-fd88-416e-ae2c-9eb488bb6cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:41.022981 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abea41-fd88-416e-ae2c-9eb488bb6cd5 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:41.029279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:41.029936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:41.030250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:41.030484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] PATH_INFO: `/v3/projects/7b6e994f692c4732a989fadcd5afe3a8` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:41.032150 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6e994f692c4732a989fadcd5afe3a8)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:41.033635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:41.157819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6e994f692c4732a989fadcd5afe3a8', 'request_id': 'req-2a98b0d9-afad-4e0c-9e3c-e0df26149046'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:41.164646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6e994f692c4732a989fadcd5afe3a8', 'request_id': 'req-2a98b0d9-afad-4e0c-9e3c-e0df26149046'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:41.284013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6e994f692c4732a989fadcd5afe3a8', 'request_id': 'req-2a98b0d9-afad-4e0c-9e3c-e0df26149046'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:41.296217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] The token cache is being invalidate because project 7b6e994f692c4732a989fadcd5afe3a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:42:41.296681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:41.297588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2a98b0d9-afad-4e0c-9e3c-e0df26149046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:41.300737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1179/2400] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:42:41 2022] DELETE /identity/v3/projects/7b6e994f692c4732a989fadcd5afe3a8 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:41.304999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29913eca-2e5a-4349-a39e-eb348bee13d4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:41.306460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29913eca-2e5a-4349-a39e-eb348bee13d4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:41.306460 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29913eca-2e5a-4349-a39e-eb348bee13d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:41.331765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-29913eca-2e5a-4349-a39e-eb348bee13d4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:41.333392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-29913eca-2e5a-4349-a39e-eb348bee13d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:41.353047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:41.397787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1222/2401] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:42:41.406426 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b70c5c-c98c-41d3-a785-af93d4fef6a7 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:41.406940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b70c5c-c98c-41d3-a785-af93d4fef6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:41.406940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b70c5c-c98c-41d3-a785-af93d4fef6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:41.428982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b2b70c5c-c98c-41d3-a785-af93d4fef6a7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:41.430176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b2b70c5c-c98c-41d3-a785-af93d4fef6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:41.438195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1180/2402] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:41.442490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29913eca-2e5a-4349-a39e-eb348bee13d4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:41.452861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29913eca-2e5a-4349-a39e-eb348bee13d4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:41.458291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab0760-814e-4e44-94de-eb01b4b9f919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:41.458890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab0760-814e-4e44-94de-eb01b4b9f919 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:41.459220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab0760-814e-4e44-94de-eb01b4b9f919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:41.459527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab0760-814e-4e44-94de-eb01b4b9f919 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:41.460187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab0760-814e-4e44-94de-eb01b4b9f919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:42:41.460490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab0760-814e-4e44-94de-eb01b4b9f919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:41.462170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab0760-814e-4e44-94de-eb01b4b9f919 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:41.475235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1223/2403] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:42:41.480354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b70c5c-c98c-41d3-a785-af93d4fef6a7 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:41.489785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b70c5c-c98c-41d3-a785-af93d4fef6a7 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:41.495146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c141c1-93ac-496e-b4dd-d912f82dea5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:41.495745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c141c1-93ac-496e-b4dd-d912f82dea5e admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:41.496027 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c141c1-93ac-496e-b4dd-d912f82dea5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:41.496309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c141c1-93ac-496e-b4dd-d912f82dea5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:41.496967 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c141c1-93ac-496e-b4dd-d912f82dea5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:41.499126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c141c1-93ac-496e-b4dd-d912f82dea5e admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:41.641593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1181/2404] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:42:41 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:42:41.646291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab0760-814e-4e44-94de-eb01b4b9f919 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:41.664590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab0760-814e-4e44-94de-eb01b4b9f919 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:41.726876 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2305bc-1f76-42ac-bfe4-2128a9a51ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:41.727482 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2305bc-1f76-42ac-bfe4-2128a9a51ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:41.727756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2305bc-1f76-42ac-bfe4-2128a9a51ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:41.728210 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2305bc-1f76-42ac-bfe4-2128a9a51ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:41.728956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2305bc-1f76-42ac-bfe4-2128a9a51ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:41.731005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2305bc-1f76-42ac-bfe4-2128a9a51ae7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:41.839029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1224/2405] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:42:41 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:42:41.843375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c141c1-93ac-496e-b4dd-d912f82dea5e admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:41.854135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c141c1-93ac-496e-b4dd-d912f82dea5e admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:41.858905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d4ac9-c949-48ce-b8d0-13e4853f3b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:41.859509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d4ac9-c949-48ce-b8d0-13e4853f3b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:41.859820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d4ac9-c949-48ce-b8d0-13e4853f3b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:41.860057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d4ac9-c949-48ce-b8d0-13e4853f3b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:41.860686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d4ac9-c949-48ce-b8d0-13e4853f3b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:41.862317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d4ac9-c949-48ce-b8d0-13e4853f3b3b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:41.874342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1182/2406] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:42:41.878990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2305bc-1f76-42ac-bfe4-2128a9a51ae7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:41.891984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2305bc-1f76-42ac-bfe4-2128a9a51ae7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:41.897262 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71dbed7-843c-4be2-8a77-732e900a07ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:41.897858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dbed7-843c-4be2-8a77-732e900a07ae admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:41.898132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dbed7-843c-4be2-8a77-732e900a07ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:41.898401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71dbed7-843c-4be2-8a77-732e900a07ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:41.899040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dbed7-843c-4be2-8a77-732e900a07ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:41.900902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71dbed7-843c-4be2-8a77-732e900a07ae admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:41.912361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1225/2407] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:42:41.917359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d4ac9-c949-48ce-b8d0-13e4853f3b3b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:41.927677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d4ac9-c949-48ce-b8d0-13e4853f3b3b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:41.933187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297da7a4-bb09-45e8-821f-fae4c64f5e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:41.933838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297da7a4-bb09-45e8-821f-fae4c64f5e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:41.934109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297da7a4-bb09-45e8-821f-fae4c64f5e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:41.934410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297da7a4-bb09-45e8-821f-fae4c64f5e41 admin admin] PATH_INFO: `/v3/projects/0d33e608f4494a1c895404afcb3f1f85/users/57246d98fbc74776a268560b410845fc/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:41.950068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297da7a4-bb09-45e8-821f-fae4c64f5e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d33e608f4494a1c895404afcb3f1f85, user_id=57246d98fbc74776a268560b410845fc, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:41.952822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297da7a4-bb09-45e8-821f-fae4c64f5e41 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:42.049383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1183/2408] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:42:41 2022] PUT /identity/v3/projects/0d33e608f4494a1c895404afcb3f1f85/users/57246d98fbc74776a268560b410845fc/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:42.066285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6392129-0916-4115-96c8-80358992736c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:42.066566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6392129-0916-4115-96c8-80358992736c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:42.066837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6392129-0916-4115-96c8-80358992736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:42.096100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c6392129-0916-4115-96c8-80358992736c None None] MFA Rules not processed for user `57246d98fbc74776a268560b410845fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:42.097344 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c6392129-0916-4115-96c8-80358992736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:42.168066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1226/2409] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:42:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:42:42.179115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297da7a4-bb09-45e8-821f-fae4c64f5e41 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:42.180873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-297da7a4-bb09-45e8-821f-fae4c64f5e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:42.204629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:42.239655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297da7a4-bb09-45e8-821f-fae4c64f5e41 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:42.243406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3c3ea-6b35-4e38-8410-f98d2634caa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:42.244580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3c3ea-6b35-4e38-8410-f98d2634caa1 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:42.244933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3c3ea-6b35-4e38-8410-f98d2634caa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:42.245202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3c3ea-6b35-4e38-8410-f98d2634caa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:42.254318 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3c3ea-6b35-4e38-8410-f98d2634caa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:42.255674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3c3ea-6b35-4e38-8410-f98d2634caa1 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:42.265594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1184/2410] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:42.292685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6392129-0916-4115-96c8-80358992736c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:42.312664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:42.371634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6392129-0916-4115-96c8-80358992736c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:42.377180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:42.377833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:42.378107 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:42.378374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] PATH_INFO: `/v3/users/57246d98fbc74776a268560b410845fc` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:42.379557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=57246d98fbc74776a268560b410845fc)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:42.381488 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:42.630655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57246d98fbc74776a268560b410845fc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:42.724052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57246d98fbc74776a268560b410845fc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:42.738439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57246d98fbc74776a268560b410845fc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:42.739206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57246d98fbc74776a268560b410845fc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:42.746730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57246d98fbc74776a268560b410845fc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:42.753434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1227/2411] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:42:42 2022] DELETE /identity/v3/users/57246d98fbc74776a268560b410845fc => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:42:42.758396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a589c-2ee1-454f-8036-28d89771cd02 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:42.758988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a589c-2ee1-454f-8036-28d89771cd02 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:42.758988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a589c-2ee1-454f-8036-28d89771cd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:42.779659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-439a589c-2ee1-454f-8036-28d89771cd02 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:42:42.780941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-439a589c-2ee1-454f-8036-28d89771cd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:42.840941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1185/2412] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:42:42.851127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:42.852648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:42.916032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06680a-e3f6-46d3-a4ed-6bcf1667fe3f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:42.919792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345b053-c91a-49cb-b50e-47a2f3afe1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:42.920398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345b053-c91a-49cb-b50e-47a2f3afe1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:42.920704 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345b053-c91a-49cb-b50e-47a2f3afe1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:42.921000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345b053-c91a-49cb-b50e-47a2f3afe1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:42.930681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345b053-c91a-49cb-b50e-47a2f3afe1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:42.932400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345b053-c91a-49cb-b50e-47a2f3afe1c7 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:42.942508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1228/2413] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:43.517151 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a589c-2ee1-454f-8036-28d89771cd02 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:43.518859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-439a589c-2ee1-454f-8036-28d89771cd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:43.534650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a589c-2ee1-454f-8036-28d89771cd02 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:43.539307 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:43.539961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:43.540261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:43.541142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] PATH_INFO: `/v3/projects/0d33e608f4494a1c895404afcb3f1f85` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:43.542371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d33e608f4494a1c895404afcb3f1f85)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:43.544336 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:43.696503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d33e608f4494a1c895404afcb3f1f85', 'request_id': 'req-4990ee4d-1708-425e-9c8b-3206cce3ebc9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:43.705510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d33e608f4494a1c895404afcb3f1f85', 'request_id': 'req-4990ee4d-1708-425e-9c8b-3206cce3ebc9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:43.781487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d33e608f4494a1c895404afcb3f1f85', 'request_id': 'req-4990ee4d-1708-425e-9c8b-3206cce3ebc9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:43.794438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] The token cache is being invalidate because project 0d33e608f4494a1c895404afcb3f1f85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:42:43.794728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:43.795385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:42:43.796985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1186/2414] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:42:43 2022] DELETE /identity/v3/projects/0d33e608f4494a1c895404afcb3f1f85 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:42:50.445899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345b053-c91a-49cb-b50e-47a2f3afe1c7 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:50.455711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345b053-c91a-49cb-b50e-47a2f3afe1c7 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:50.503720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:50.519033 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff255a-4b91-4413-a497-1172343a67b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:50.519760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff255a-4b91-4413-a497-1172343a67b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:50.520043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff255a-4b91-4413-a497-1172343a67b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:50.520312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff255a-4b91-4413-a497-1172343a67b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:50.522184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a8ff255a-4b91-4413-a497-1172343a67b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:50.533351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff255a-4b91-4413-a497-1172343a67b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:50.534781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff255a-4b91-4413-a497-1172343a67b3 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:50.543060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1229/2415] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:51.233515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:51.235061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:51.301308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:42:51.329183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990ee4d-1708-425e-9c8b-3206cce3ebc9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:51.336624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa3eed-73f4-4984-908f-b0482c46b54b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:51.344256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa3eed-73f4-4984-908f-b0482c46b54b service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:51.344256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa3eed-73f4-4984-908f-b0482c46b54b service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:51.344256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa3eed-73f4-4984-908f-b0482c46b54b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:51.344256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c8aa3eed-73f4-4984-908f-b0482c46b54b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:51.443870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa3eed-73f4-4984-908f-b0482c46b54b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:51.445942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa3eed-73f4-4984-908f-b0482c46b54b service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:51.470387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1187/2416] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:42:52.403817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff255a-4b91-4413-a497-1172343a67b3 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:52.425473 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff255a-4b91-4413-a497-1172343a67b3 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:52.431815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:52.433151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:52.433429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:52.433706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:52.437570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:52.546510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:52.548230 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:52.570579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1230/2417] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:42:55.295357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa3eed-73f4-4984-908f-b0482c46b54b service nova] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:42:55.311930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa3eed-73f4-4984-908f-b0482c46b54b service nova] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:42:55.317890 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36437676-2514-4ea6-8d02-3a8af8dbcb7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:42:55.318505 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36437676-2514-4ea6-8d02-3a8af8dbcb7c service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:42:55.318785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36437676-2514-4ea6-8d02-3a8af8dbcb7c service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:42:55.319054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36437676-2514-4ea6-8d02-3a8af8dbcb7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:42:55.320912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-36437676-2514-4ea6-8d02-3a8af8dbcb7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:42:55.335052 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36437676-2514-4ea6-8d02-3a8af8dbcb7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:42:55.336570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36437676-2514-4ea6-8d02-3a8af8dbcb7c service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:42:55.350329 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1188/2418] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:42:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:02.567599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=49 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:05.344822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=54 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:11.024247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:11.027109 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:11.044887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae6218-bb11-4f6c-a887-8da9c0cb08cd service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:11.052319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:11.053182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:11.053553 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:11.053906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:11.056226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:11.072479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:11.074508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:11.089420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1231/2419] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:19.152985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b092e24-fdb8-4754-956b-95239e4fb007 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:19.158145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b092e24-fdb8-4754-956b-95239e4fb007 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:19.158145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b092e24-fdb8-4754-956b-95239e4fb007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:19.183598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-8b092e24-fdb8-4754-956b-95239e4fb007 None None] MFA Rules not processed for user `2715cf2a5129439c83a5347bb963f1a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:19.184847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8b092e24-fdb8-4754-956b-95239e4fb007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:19.193141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1189/2420] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:43:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:19.528788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:19.530566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:19.549747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc7993-f3c9-49f8-a9ca-ce792fac3e1e service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:19.554289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:19.555092 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:19.572237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:19.572237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:19.572237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:19.573248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:19.574864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:19.584043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1232/2421] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:19.999594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9d950-5a2d-40d3-928d-f390cb15b4cb None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:19.999874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9d950-5a2d-40d3-928d-f390cb15b4cb None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:20.000145 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9d950-5a2d-40d3-928d-f390cb15b4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:20.097870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e4a9d950-5a2d-40d3-928d-f390cb15b4cb None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:20.099090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e4a9d950-5a2d-40d3-928d-f390cb15b4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:20.108052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:20.109480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:20.192125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1190/2422] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:43:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:20.203874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252cd6d-13c2-4539-a5cb-33c84ebdf98b service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:20.209501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34175350-b020-493a-b4aa-db463f22e335 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:20.210168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34175350-b020-493a-b4aa-db463f22e335 service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:20.210992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34175350-b020-493a-b4aa-db463f22e335 service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:20.211275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34175350-b020-493a-b4aa-db463f22e335 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:20.219725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-34175350-b020-493a-b4aa-db463f22e335 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:20.231844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34175350-b020-493a-b4aa-db463f22e335 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:20.233464 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34175350-b020-493a-b4aa-db463f22e335 service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:20.244902 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1233/2423] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:43:20.352782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9d950-5a2d-40d3-928d-f390cb15b4cb None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:20.354860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e4a9d950-5a2d-40d3-928d-f390cb15b4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:20.459597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9d950-5a2d-40d3-928d-f390cb15b4cb None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:20.465950 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96887bf9-da47-46a1-902f-a56d650191b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:20.466566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96887bf9-da47-46a1-902f-a56d650191b6 service swift] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:20.466843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96887bf9-da47-46a1-902f-a56d650191b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:20.467110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96887bf9-da47-46a1-902f-a56d650191b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:20.476523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96887bf9-da47-46a1-902f-a56d650191b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:20.478202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96887bf9-da47-46a1-902f-a56d650191b6 service swift] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:20.486875 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1191/2424] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:43:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:43:30.270783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=44 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:30.483579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:39.651537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34175350-b020-493a-b4aa-db463f22e335 service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:39.664848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34175350-b020-493a-b4aa-db463f22e335 service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:39.671227 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:39.672095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:39.672506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:39.673002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:39.675205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:39.691205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:39.698498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:39.717542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1234/2425] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:43:45.534500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96887bf9-da47-46a1-902f-a56d650191b6 service swift] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:45.536237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-96887bf9-da47-46a1-902f-a56d650191b6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:45.642271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96887bf9-da47-46a1-902f-a56d650191b6 service swift] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:45.645871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113975d-6417-4a49-990f-e29c8286cebc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ac781cf2d84d95a789724e74e12acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:45.646472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113975d-6417-4a49-990f-e29c8286cebc service glance] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:45.646759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113975d-6417-4a49-990f-e29c8286cebc service glance] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:45.647026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113975d-6417-4a49-990f-e29c8286cebc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:45.648750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a113975d-6417-4a49-990f-e29c8286cebc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:45.666089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113975d-6417-4a49-990f-e29c8286cebc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:45.667723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113975d-6417-4a49-990f-e29c8286cebc service glance] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:45.679057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1192/2426] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:45.934303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:45.935940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:45.954795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fdeb1-4105-46cd-b032-f931081f3a8e service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:45.964911 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff6b1-68db-4329-ab5d-f30fde861b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:45.966576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff6b1-68db-4329-ab5d-f30fde861b94 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:45.967924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff6b1-68db-4329-ab5d-f30fde861b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:45.968706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aff6b1-68db-4329-ab5d-f30fde861b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:45.973394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-88aff6b1-68db-4329-ab5d-f30fde861b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:45.984862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aff6b1-68db-4329-ab5d-f30fde861b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:45.986734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aff6b1-68db-4329-ab5d-f30fde861b94 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:45.997424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1235/2427] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:46.075007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113975d-6417-4a49-990f-e29c8286cebc service glance] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:46.084903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113975d-6417-4a49-990f-e29c8286cebc service glance] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:46.089583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cf970-9e5d-423d-a655-b14ba11ac298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:46.090180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cf970-9e5d-423d-a655-b14ba11ac298 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:46.090448 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cf970-9e5d-423d-a655-b14ba11ac298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:46.090718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cf970-9e5d-423d-a655-b14ba11ac298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:46.092492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ca5cf970-9e5d-423d-a655-b14ba11ac298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:46.111242 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cf970-9e5d-423d-a655-b14ba11ac298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:46.114169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cf970-9e5d-423d-a655-b14ba11ac298 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:46.126874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1193/2428] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:48.124639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff6b1-68db-4329-ab5d-f30fde861b94 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:48.139689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aff6b1-68db-4329-ab5d-f30fde861b94 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:48.145842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c13795-a24c-422a-8288-97f3c91be9a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:48.146513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c13795-a24c-422a-8288-97f3c91be9a2 service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:48.146827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c13795-a24c-422a-8288-97f3c91be9a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:48.147180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c13795-a24c-422a-8288-97f3c91be9a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:48.150409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e3c13795-a24c-422a-8288-97f3c91be9a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:48.236695 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c13795-a24c-422a-8288-97f3c91be9a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:48.238522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c13795-a24c-422a-8288-97f3c91be9a2 service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:48.278601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1236/2429] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:43:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:43:49.361570 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e2b9e-cea7-44fb-9607-213e57e60d85 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.365209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e2b9e-cea7-44fb-9607-213e57e60d85 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.365209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e2b9e-cea7-44fb-9607-213e57e60d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.383202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-f62e2b9e-cea7-44fb-9607-213e57e60d85 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:43:49.385064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f62e2b9e-cea7-44fb-9607-213e57e60d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:43:49.393472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1194/2430] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:43:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:43:49.406413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c13795-a24c-422a-8288-97f3c91be9a2 service nova] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:43:49.420950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c13795-a24c-422a-8288-97f3c91be9a2 service nova] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:43:49.428557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b675901-00fd-4644-814e-e24527b87968 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:43:49.429504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b675901-00fd-4644-814e-e24527b87968 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:43:49.429985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b675901-00fd-4644-814e-e24527b87968 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:43:49.430414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b675901-00fd-4644-814e-e24527b87968 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:43:49.439752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b675901-00fd-4644-814e-e24527b87968 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:43:49.441358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b675901-00fd-4644-814e-e24527b87968 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:43:49.449692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1237/2431] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:43:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:43:59.377529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=44 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:43:59.448065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=50 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:14.552352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51089e-e139-4be7-9842-3378b08bddc5 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:14.552963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51089e-e139-4be7-9842-3378b08bddc5 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:14.552963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51089e-e139-4be7-9842-3378b08bddc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:14.574540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-fd51089e-e139-4be7-9842-3378b08bddc5 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:44:14.575839 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fd51089e-e139-4be7-9842-3378b08bddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:14.583524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1195/2432] 10.222.0.84 () {58 vars in 967 bytes} [Wed Aug 24 11:44:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:44:14.597139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b675901-00fd-4644-814e-e24527b87968 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:14.608510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b675901-00fd-4644-814e-e24527b87968 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:14.618567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb307f-4019-475c-82bc-7b9efa0adb66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:14.619176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb307f-4019-475c-82bc-7b9efa0adb66 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:14.619451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb307f-4019-475c-82bc-7b9efa0adb66 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:14.620235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb307f-4019-475c-82bc-7b9efa0adb66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:14.629930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb307f-4019-475c-82bc-7b9efa0adb66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:14.631515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb307f-4019-475c-82bc-7b9efa0adb66 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:14.641762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1238/2433] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:44:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:44:24.568121 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:24.637980 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=6 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:29.756476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c65b49-efc4-4a49-a9fa-e4babef55a19 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:29.757186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c65b49-efc4-4a49-a9fa-e4babef55a19 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:29.757186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c65b49-efc4-4a49-a9fa-e4babef55a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:29.776051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e6c65b49-efc4-4a49-a9fa-e4babef55a19 None None] MFA Rules not processed for user `0b82b6f568bd41fa9d6b44facb766e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:44:29.777305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e6c65b49-efc4-4a49-a9fa-e4babef55a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:29.839592 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1196/2434] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:44:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:44:30.185453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb307f-4019-475c-82bc-7b9efa0adb66 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:30.196535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb307f-4019-475c-82bc-7b9efa0adb66 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:30.201298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11f5e2-2a3c-4c42-aae2-f38b62b2dc9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:30.201918 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11f5e2-2a3c-4c42-aae2-f38b62b2dc9e service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:30.202183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11f5e2-2a3c-4c42-aae2-f38b62b2dc9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:30.202472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11f5e2-2a3c-4c42-aae2-f38b62b2dc9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:30.204282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6f11f5e2-2a3c-4c42-aae2-f38b62b2dc9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:30.214884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11f5e2-2a3c-4c42-aae2-f38b62b2dc9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:30.217200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11f5e2-2a3c-4c42-aae2-f38b62b2dc9e service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:30.226849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1239/2435] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:44:39.832842 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=25 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:40.222524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=6 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:44:55.429442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c65b49-efc4-4a49-a9fa-e4babef55a19 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:55.439144 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c65b49-efc4-4a49-a9fa-e4babef55a19 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:55.442823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f71fe-48b5-4164-ae62-9e9a8049d9c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:55.443413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f71fe-48b5-4164-ae62-9e9a8049d9c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:55.443687 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f71fe-48b5-4164-ae62-9e9a8049d9c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:55.443955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f71fe-48b5-4164-ae62-9e9a8049d9c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:55.445941 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-921f71fe-48b5-4164-ae62-9e9a8049d9c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:55.511331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f71fe-48b5-4164-ae62-9e9a8049d9c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:55.512754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f71fe-48b5-4164-ae62-9e9a8049d9c5 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:55.522807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1197/2436] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:44:56.791406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11f5e2-2a3c-4c42-aae2-f38b62b2dc9e service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:56.802497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11f5e2-2a3c-4c42-aae2-f38b62b2dc9e service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:56.806258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:56.807079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:56.807351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:56.807621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:56.809424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:56.877804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:56.879268 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:56.889664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1240/2437] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:44:59.202520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f71fe-48b5-4164-ae62-9e9a8049d9c5 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:44:59.215010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f71fe-48b5-4164-ae62-9e9a8049d9c5 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:44:59.220640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:44:59.221296 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:44:59.221758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:44:59.222032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:44:59.223801 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:44:59.312652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:44:59.315286 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:44:59.330215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1198/2438] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:45:04.590229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:04.592350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:04.608162 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d5122-4fd8-48b6-8c8f-4ccfe366a005 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:04.613698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4974663-71e8-4891-8ba2-b4236013fa6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:04.617640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4974663-71e8-4891-8ba2-b4236013fa6e service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:04.617640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4974663-71e8-4891-8ba2-b4236013fa6e service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:04.617640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4974663-71e8-4891-8ba2-b4236013fa6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:04.617640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d4974663-71e8-4891-8ba2-b4236013fa6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:04.641723 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4974663-71e8-4891-8ba2-b4236013fa6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:04.643317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4974663-71e8-4891-8ba2-b4236013fa6e service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:04.654719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1241/2439] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:45:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:45:04.720779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:04.722837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:04.741841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec57b7-05b6-480c-8a11-5d0893ac5c12 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:04.747147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:04.747876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:04.748149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:04.748415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] PATH_INFO: `/v3/users/0031939e16194f63b99e6f27f700fa8c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:04.749679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0031939e16194f63b99e6f27f700fa8c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:04.752594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:05.017598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0031939e16194f63b99e6f27f700fa8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:05.018353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0031939e16194f63b99e6f27f700fa8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:05.018969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0031939e16194f63b99e6f27f700fa8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:05.026028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0031939e16194f63b99e6f27f700fa8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:05.043360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0031939e16194f63b99e6f27f700fa8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:05.152525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:05.157562 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1199/2440] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:45:04 2022] DELETE /identity/v3/users/0031939e16194f63b99e6f27f700fa8c => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:45:05.254853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4974663-71e8-4891-8ba2-b4236013fa6e service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:05.265328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4974663-71e8-4891-8ba2-b4236013fa6e service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:05.320694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67287b67-6acc-4f2a-a4ca-b71f02f58b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:05.321340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287b67-6acc-4f2a-a4ca-b71f02f58b44 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:05.321615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287b67-6acc-4f2a-a4ca-b71f02f58b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:05.321891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67287b67-6acc-4f2a-a4ca-b71f02f58b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:05.323681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-67287b67-6acc-4f2a-a4ca-b71f02f58b44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:05.395672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67287b67-6acc-4f2a-a4ca-b71f02f58b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:05.397273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67287b67-6acc-4f2a-a4ca-b71f02f58b44 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:05.407364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1242/2441] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:45:05.616812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:05.618609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:05.647575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cc02e-ed01-4149-a2ba-8c86493ac2f1 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:05.659210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:05.659210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:05.659210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:05.659210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] PATH_INFO: `/v3/projects/ecec6d0c155f4d658197c4779ab9ffce` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:05.660202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecec6d0c155f4d658197c4779ab9ffce)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:05.663276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:05.880047 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecec6d0c155f4d658197c4779ab9ffce', 'request_id': 'req-bb2e881a-1888-4f7e-8317-6a01f96def71'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:05.886977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecec6d0c155f4d658197c4779ab9ffce', 'request_id': 'req-bb2e881a-1888-4f7e-8317-6a01f96def71'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:06.029989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecec6d0c155f4d658197c4779ab9ffce', 'request_id': 'req-bb2e881a-1888-4f7e-8317-6a01f96def71'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:06.043486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] The token cache is being invalidate because project ecec6d0c155f4d658197c4779ab9ffce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:45:06.043760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:06.044582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-bb2e881a-1888-4f7e-8317-6a01f96def71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:06.046996 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1200/2442] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:45:05 2022] DELETE /identity/v3/projects/ecec6d0c155f4d658197c4779ab9ffce => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:45:06.051728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527366f-13e6-42d9-86ac-4142395260a9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:06.052328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527366f-13e6-42d9-86ac-4142395260a9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:06.052328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527366f-13e6-42d9-86ac-4142395260a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:06.073730 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-c527366f-13e6-42d9-86ac-4142395260a9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:45:06.074969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c527366f-13e6-42d9-86ac-4142395260a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:06.110657 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:06.143482 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1243/2443] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:45:06.152553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e976c-58a3-414c-911e-05023d872c3e None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:06.152895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e976c-58a3-414c-911e-05023d872c3e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:06.153172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e976c-58a3-414c-911e-05023d872c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:06.176136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-977e976c-58a3-414c-911e-05023d872c3e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:45:06.177456 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-977e976c-58a3-414c-911e-05023d872c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:06.194146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1201/2444] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:45:06.198277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527366f-13e6-42d9-86ac-4142395260a9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:06.209358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527366f-13e6-42d9-86ac-4142395260a9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:06.214384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1379-f8e1-46a5-90c4-2f595c0191e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:06.215015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1379-f8e1-46a5-90c4-2f595c0191e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:06.215292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1379-f8e1-46a5-90c4-2f595c0191e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:06.215566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1379-f8e1-46a5-90c4-2f595c0191e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:06.216259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb1379-f8e1-46a5-90c4-2f595c0191e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:45:06.217273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb1379-f8e1-46a5-90c4-2f595c0191e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:06.218995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb1379-f8e1-46a5-90c4-2f595c0191e2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:06.232855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1244/2445] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:45:06.238134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e976c-58a3-414c-911e-05023d872c3e None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:06.248470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e976c-58a3-414c-911e-05023d872c3e None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:06.253712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ceb44-b695-4fec-b4cd-46eec5868e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:06.254334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ceb44-b695-4fec-b4cd-46eec5868e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:06.254613 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ceb44-b695-4fec-b4cd-46eec5868e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:06.254884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ceb44-b695-4fec-b4cd-46eec5868e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:06.255517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ceb44-b695-4fec-b4cd-46eec5868e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:06.257517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ceb44-b695-4fec-b4cd-46eec5868e07 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:06.363496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1202/2446] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:45:06 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:45:06.367586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1379-f8e1-46a5-90c4-2f595c0191e2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:06.389662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1379-f8e1-46a5-90c4-2f595c0191e2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:06.451625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590275-dc44-442d-b5c3-e0d74de26020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:06.452221 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590275-dc44-442d-b5c3-e0d74de26020 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:06.452493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590275-dc44-442d-b5c3-e0d74de26020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:06.452762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10590275-dc44-442d-b5c3-e0d74de26020 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:06.453422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10590275-dc44-442d-b5c3-e0d74de26020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:06.455338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10590275-dc44-442d-b5c3-e0d74de26020 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:06.585915 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1245/2447] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:45:06 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:45:06.591085 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ceb44-b695-4fec-b4cd-46eec5868e07 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:06.602297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ceb44-b695-4fec-b4cd-46eec5868e07 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:06.607931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18969043-3602-4f55-ae28-f9449881aa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:06.608751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18969043-3602-4f55-ae28-f9449881aa9b admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:06.609057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18969043-3602-4f55-ae28-f9449881aa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:06.609322 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18969043-3602-4f55-ae28-f9449881aa9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:06.609965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18969043-3602-4f55-ae28-f9449881aa9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:06.611753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18969043-3602-4f55-ae28-f9449881aa9b admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:06.624581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1203/2448] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:45:06.628892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590275-dc44-442d-b5c3-e0d74de26020 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:06.639620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10590275-dc44-442d-b5c3-e0d74de26020 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:06.644489 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4d857-567b-42a2-af72-88860a6a8a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:06.645116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4d857-567b-42a2-af72-88860a6a8a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:06.645410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4d857-567b-42a2-af72-88860a6a8a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:06.645666 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4d857-567b-42a2-af72-88860a6a8a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:06.646431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4d857-567b-42a2-af72-88860a6a8a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:06.647937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4d857-567b-42a2-af72-88860a6a8a23 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:06.663851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1246/2449] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:45:06.668277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18969043-3602-4f55-ae28-f9449881aa9b admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:06.679750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18969043-3602-4f55-ae28-f9449881aa9b admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:06.685633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad381681-705b-48ef-a981-5326c14b721e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:06.686745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381681-705b-48ef-a981-5326c14b721e admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:06.687032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381681-705b-48ef-a981-5326c14b721e admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:06.687421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381681-705b-48ef-a981-5326c14b721e admin admin] PATH_INFO: `/v3/projects/51a4be7fb0b94dafaf9f583050eb1508/users/3aeb2625b5514e9fb9ca8dbc3097d3f6/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:06.708064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad381681-705b-48ef-a981-5326c14b721e admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a4be7fb0b94dafaf9f583050eb1508, user_id=3aeb2625b5514e9fb9ca8dbc3097d3f6, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:06.710857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad381681-705b-48ef-a981-5326c14b721e admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:06.823122 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1204/2450] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:45:06 2022] PUT /identity/v3/projects/51a4be7fb0b94dafaf9f583050eb1508/users/3aeb2625b5514e9fb9ca8dbc3097d3f6/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:45:06.825228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d79907-5619-4cce-8e45-7617f2fe2821 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:06.826689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d79907-5619-4cce-8e45-7617f2fe2821 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:06.827246 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d79907-5619-4cce-8e45-7617f2fe2821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:06.864220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e2d79907-5619-4cce-8e45-7617f2fe2821 None None] MFA Rules not processed for user `3aeb2625b5514e9fb9ca8dbc3097d3f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:45:06.865815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e2d79907-5619-4cce-8e45-7617f2fe2821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:06.970397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1247/2451] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:45:06.982633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad381681-705b-48ef-a981-5326c14b721e admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:06.984268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ad381681-705b-48ef-a981-5326c14b721e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:07.073308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad381681-705b-48ef-a981-5326c14b721e admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:07.079602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8daba-1159-420d-837e-12a26114bfa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:07.080221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8daba-1159-420d-837e-12a26114bfa1 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:07.080485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8daba-1159-420d-837e-12a26114bfa1 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:07.080751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8daba-1159-420d-837e-12a26114bfa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:07.093007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8daba-1159-420d-837e-12a26114bfa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:07.094947 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8daba-1159-420d-837e-12a26114bfa1 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:07.105115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1205/2452] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:45:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:45:11.754506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d79907-5619-4cce-8e45-7617f2fe2821 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:11.801550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:11.840927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d79907-5619-4cce-8e45-7617f2fe2821 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:11.845844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:11.846471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:11.846727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:11.846996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] PATH_INFO: `/v3/users/3aeb2625b5514e9fb9ca8dbc3097d3f6` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:11.848167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aeb2625b5514e9fb9ca8dbc3097d3f6)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:11.850126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:12.070406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aeb2625b5514e9fb9ca8dbc3097d3f6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:12.145359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aeb2625b5514e9fb9ca8dbc3097d3f6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:12.170187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aeb2625b5514e9fb9ca8dbc3097d3f6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:12.170949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aeb2625b5514e9fb9ca8dbc3097d3f6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:12.178728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aeb2625b5514e9fb9ca8dbc3097d3f6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:12.191506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dfd29-c1a7-4a44-a72d-8411fa7c1979 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:12.191999 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dfd29-c1a7-4a44-a72d-8411fa7c1979 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:12.192083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dfd29-c1a7-4a44-a72d-8411fa7c1979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:12.198846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1248/2453] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:45:11 2022] DELETE /identity/v3/users/3aeb2625b5514e9fb9ca8dbc3097d3f6 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:45:12.204343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:12.217488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-fa1dfd29-c1a7-4a44-a72d-8411fa7c1979 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:45:12.218736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fa1dfd29-c1a7-4a44-a72d-8411fa7c1979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:12.286917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1206/2454] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:45:12.298159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:12.299623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:12.374329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819b467-b56c-4b7d-b7b1-8c65c06b5b72 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:12.378283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a614b-d5b0-47e5-bdd0-f6a34801c096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:12.378898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a614b-d5b0-47e5-bdd0-f6a34801c096 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:12.379195 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a614b-d5b0-47e5-bdd0-f6a34801c096 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:12.379468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a614b-d5b0-47e5-bdd0-f6a34801c096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:12.389872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a614b-d5b0-47e5-bdd0-f6a34801c096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:12.392534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a614b-d5b0-47e5-bdd0-f6a34801c096 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:12.409717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1249/2455] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:45:13.042867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dfd29-c1a7-4a44-a72d-8411fa7c1979 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:13.044347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fa1dfd29-c1a7-4a44-a72d-8411fa7c1979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:13.063187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dfd29-c1a7-4a44-a72d-8411fa7c1979 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:13.068646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:13.069368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:13.069643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:13.070135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] PATH_INFO: `/v3/projects/51a4be7fb0b94dafaf9f583050eb1508` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:13.071312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a4be7fb0b94dafaf9f583050eb1508)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:13.073490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:13.216295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a4be7fb0b94dafaf9f583050eb1508', 'request_id': 'req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:13.225383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a4be7fb0b94dafaf9f583050eb1508', 'request_id': 'req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:13.350773 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a4be7fb0b94dafaf9f583050eb1508', 'request_id': 'req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:13.364668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] The token cache is being invalidate because project 51a4be7fb0b94dafaf9f583050eb1508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:45:13.365055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:13.367728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ca2ed9a6-2e5e-406d-8a09-a9e4fbb48252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:13.370540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1207/2456] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:45:13 2022] DELETE /identity/v3/projects/51a4be7fb0b94dafaf9f583050eb1508 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:45:13.382988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96090922-4631-4a79-a6c7-b4ac32119ab9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:13.383467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96090922-4631-4a79-a6c7-b4ac32119ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:13.383576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96090922-4631-4a79-a6c7-b4ac32119ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:13.409527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-96090922-4631-4a79-a6c7-b4ac32119ab9 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:45:13.410889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-96090922-4631-4a79-a6c7-b4ac32119ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:13.465884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:13.480364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1250/2457] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:45:13.489532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdf3c9-e81a-44b4-9fbf-b7a2272772d7 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:13.490595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdf3c9-e81a-44b4-9fbf-b7a2272772d7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:13.490595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdf3c9-e81a-44b4-9fbf-b7a2272772d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:13.511474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-f4bdf3c9-e81a-44b4-9fbf-b7a2272772d7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:45:13.512620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f4bdf3c9-e81a-44b4-9fbf-b7a2272772d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:13.521772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1208/2458] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:45:13.526397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96090922-4631-4a79-a6c7-b4ac32119ab9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:13.537903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96090922-4631-4a79-a6c7-b4ac32119ab9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:13.543012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacd95d-a997-4df1-9f92-204455941466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:13.543727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd95d-a997-4df1-9f92-204455941466 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:13.544950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd95d-a997-4df1-9f92-204455941466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:13.545287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd95d-a997-4df1-9f92-204455941466 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:13.546087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacd95d-a997-4df1-9f92-204455941466 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:45:13.546465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacd95d-a997-4df1-9f92-204455941466 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:13.549085 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacd95d-a997-4df1-9f92-204455941466 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:13.570675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1251/2459] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:45:13.570822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdf3c9-e81a-44b4-9fbf-b7a2272772d7 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:13.590140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdf3c9-e81a-44b4-9fbf-b7a2272772d7 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:13.596450 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063f5ec-652a-4381-a141-903614083f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:13.597042 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063f5ec-652a-4381-a141-903614083f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:13.597969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063f5ec-652a-4381-a141-903614083f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:13.598635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063f5ec-652a-4381-a141-903614083f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:13.599436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063f5ec-652a-4381-a141-903614083f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:13.602128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063f5ec-652a-4381-a141-903614083f42 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:13.733851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1209/2460] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:45:13 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:45:13.735077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacd95d-a997-4df1-9f92-204455941466 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:13.748187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacd95d-a997-4df1-9f92-204455941466 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:13.815449 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbac34f-ca51-426c-8ab6-d62fab785ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:13.816057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbac34f-ca51-426c-8ab6-d62fab785ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:13.816396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbac34f-ca51-426c-8ab6-d62fab785ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:13.816676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbac34f-ca51-426c-8ab6-d62fab785ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:13.817325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbac34f-ca51-426c-8ab6-d62fab785ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:13.819274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbac34f-ca51-426c-8ab6-d62fab785ee6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:13.918375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063f5ec-652a-4381-a141-903614083f42 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:13.921135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8063f5ec-652a-4381-a141-903614083f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:13.941916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063f5ec-652a-4381-a141-903614083f42 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:13.948270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:13.952773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1252/2461] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:45:13 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:45:13.952915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:13.952915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:13.952915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] PATH_INFO: `/v3/users/7ac5dffd3204413a8834d05db267c3ab` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:13.952915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac5dffd3204413a8834d05db267c3ab)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:13.953855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbac34f-ca51-426c-8ab6-d62fab785ee6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:13.954141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:13.964662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbac34f-ca51-426c-8ab6-d62fab785ee6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:13.970932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1a972-8419-489d-acd1-a834e3cd4c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:13.971790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1a972-8419-489d-acd1-a834e3cd4c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:13.971790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1a972-8419-489d-acd1-a834e3cd4c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:13.972053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1a972-8419-489d-acd1-a834e3cd4c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:13.973059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1a972-8419-489d-acd1-a834e3cd4c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:13.974606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1a972-8419-489d-acd1-a834e3cd4c37 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:13.986390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1253/2462] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:45:13.991752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1a972-8419-489d-acd1-a834e3cd4c37 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:14.003334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1a972-8419-489d-acd1-a834e3cd4c37 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:14.010785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55cd0-86d2-42f6-8e3b-0be06557dd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:14.010785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55cd0-86d2-42f6-8e3b-0be06557dd1c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:14.010785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55cd0-86d2-42f6-8e3b-0be06557dd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:14.010785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55cd0-86d2-42f6-8e3b-0be06557dd1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:14.011240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55cd0-86d2-42f6-8e3b-0be06557dd1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:14.013834 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55cd0-86d2-42f6-8e3b-0be06557dd1c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:14.024919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1254/2463] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:45:14.029499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55cd0-86d2-42f6-8e3b-0be06557dd1c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:14.041177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55cd0-86d2-42f6-8e3b-0be06557dd1c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:14.044675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672cbdf-64cb-43bd-a8a7-ae4079bae5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:14.045520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672cbdf-64cb-43bd-a8a7-ae4079bae5a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:14.045797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672cbdf-64cb-43bd-a8a7-ae4079bae5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:14.046116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672cbdf-64cb-43bd-a8a7-ae4079bae5a8 admin admin] PATH_INFO: `/v3/projects/00a60cf91e794dcba7c9ef68349b22f0/users/9d9823f2dd034a2bbb21b3f41a4292ea/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:14.062807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672cbdf-64cb-43bd-a8a7-ae4079bae5a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a60cf91e794dcba7c9ef68349b22f0, user_id=9d9823f2dd034a2bbb21b3f41a4292ea, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:14.065617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672cbdf-64cb-43bd-a8a7-ae4079bae5a8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:14.231198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1255/2464] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:45:14 2022] PUT /identity/v3/projects/00a60cf91e794dcba7c9ef68349b22f0/users/9d9823f2dd034a2bbb21b3f41a4292ea/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:45:14.242775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cae348-ffd6-4796-8fb6-bf3985e03cbd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:14.243105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cae348-ffd6-4796-8fb6-bf3985e03cbd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:14.243380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cae348-ffd6-4796-8fb6-bf3985e03cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:14.275592 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-01cae348-ffd6-4796-8fb6-bf3985e03cbd None None] MFA Rules not processed for user `9d9823f2dd034a2bbb21b3f41a4292ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:45:14.277211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-01cae348-ffd6-4796-8fb6-bf3985e03cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:14.279296 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac5dffd3204413a8834d05db267c3ab'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:14.280159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac5dffd3204413a8834d05db267c3ab'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:14.280879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac5dffd3204413a8834d05db267c3ab'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:14.286716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:14.288605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac5dffd3204413a8834d05db267c3ab'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:14.301453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac5dffd3204413a8834d05db267c3ab'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:14.346515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1256/2465] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:45:14.350904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cae348-ffd6-4796-8fb6-bf3985e03cbd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:14.352183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-01cae348-ffd6-4796-8fb6-bf3985e03cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:14.367220 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:14.392055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1210/2466] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:45:13 2022] DELETE /identity/v3/users/7ac5dffd3204413a8834d05db267c3ab => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:45:14.428706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cae348-ffd6-4796-8fb6-bf3985e03cbd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:14.501045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6d80d-e1eb-4cfa-b2d2-0973a9708441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:14.501781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6d80d-e1eb-4cfa-b2d2-0973a9708441 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:14.502057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6d80d-e1eb-4cfa-b2d2-0973a9708441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:14.502397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6d80d-e1eb-4cfa-b2d2-0973a9708441 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:14.507401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6d80d-e1eb-4cfa-b2d2-0973a9708441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:14.507401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6d80d-e1eb-4cfa-b2d2-0973a9708441 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:14.658392 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1257/2467] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:45:14 2022] POST /identity/v3/projects => generated 373 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:45:14.663589 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:14.675573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948163ab-2ed3-4215-b51e-7e7d116f707a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:14.693503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6d80d-e1eb-4cfa-b2d2-0973a9708441 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:14.696171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f3c6d80d-e1eb-4cfa-b2d2-0973a9708441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:14.768181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf2a56-9ed3-4666-90af-ab94036f1ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:14.768181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf2a56-9ed3-4666-90af-ab94036f1ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:14.768181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf2a56-9ed3-4666-90af-ab94036f1ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:14.769777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf2a56-9ed3-4666-90af-ab94036f1ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:14.770175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf2a56-9ed3-4666-90af-ab94036f1ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:14.772078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf2a56-9ed3-4666-90af-ab94036f1ee5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:14.777412 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6d80d-e1eb-4cfa-b2d2-0973a9708441 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:14.785279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:14.785937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:14.786205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:14.786476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] PATH_INFO: `/v3/projects/31998483af514a1a8f87ab761d5cf3ba` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:14.787679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31998483af514a1a8f87ab761d5cf3ba)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:14.789636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:14.924076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1211/2468] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:45:14 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:45:14.928090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf2a56-9ed3-4666-90af-ab94036f1ee5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:14.939796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf2a56-9ed3-4666-90af-ab94036f1ee5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:14.945373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287b4f-61c5-40ad-9727-def016005099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:14.946083 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287b4f-61c5-40ad-9727-def016005099 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:14.946312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287b4f-61c5-40ad-9727-def016005099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:14.946581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287b4f-61c5-40ad-9727-def016005099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:14.947217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43287b4f-61c5-40ad-9727-def016005099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:14.948927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43287b4f-61c5-40ad-9727-def016005099 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:14.967126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1212/2469] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:45:14.968642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287b4f-61c5-40ad-9727-def016005099 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:14.971376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:14.973063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31998483af514a1a8f87ab761d5cf3ba', 'request_id': 'req-c475e4f0-c1ea-431f-ac43-cf75f24db652'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:14.979015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287b4f-61c5-40ad-9727-def016005099 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:14.979828 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31998483af514a1a8f87ab761d5cf3ba', 'request_id': 'req-c475e4f0-c1ea-431f-ac43-cf75f24db652'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:15.054414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3082c2-6145-4be8-b57c-63fc0476f502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:15.055160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3082c2-6145-4be8-b57c-63fc0476f502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:15.055437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3082c2-6145-4be8-b57c-63fc0476f502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:15.055708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3082c2-6145-4be8-b57c-63fc0476f502 admin admin] PATH_INFO: `/v3/projects/917dc238190c42ea88ead51c37a58268/users/04bfd320ceca446fb284aad3ba4bd6bc/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:15.077745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3082c2-6145-4be8-b57c-63fc0476f502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=917dc238190c42ea88ead51c37a58268, user_id=04bfd320ceca446fb284aad3ba4bd6bc, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:15.080443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3082c2-6145-4be8-b57c-63fc0476f502 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:15.081999 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31998483af514a1a8f87ab761d5cf3ba', 'request_id': 'req-c475e4f0-c1ea-431f-ac43-cf75f24db652'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:15.100068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] The token cache is being invalidate because project 31998483af514a1a8f87ab761d5cf3ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:45:15.100340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:15.101287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:15.102807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1258/2470] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:45:14 2022] DELETE /identity/v3/projects/31998483af514a1a8f87ab761d5cf3ba => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:45:15.107975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:15.109546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:15.124586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475e4f0-c1ea-431f-ac43-cf75f24db652 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:15.129826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:15.130466 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:15.130743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:15.142477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] PATH_INFO: `/v3/users/353863ab3a164b9fa49644358223096a` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:15.142477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] RBAC: Authorizing `identity:delete_user(user_id=353863ab3a164b9fa49644358223096a)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:15.142477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:15.205306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1213/2471] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:45:14 2022] PUT /identity/v3/projects/917dc238190c42ea88ead51c37a58268/users/04bfd320ceca446fb284aad3ba4bd6bc/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:45:15.205306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3082c2-6145-4be8-b57c-63fc0476f502 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:15.205306 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0a3082c2-6145-4be8-b57c-63fc0476f502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:15.260920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:15.297705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3082c2-6145-4be8-b57c-63fc0476f502 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:15.302695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d484e3a-0f6b-4aa4-bf0f-8cf49f386dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:15.303280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d484e3a-0f6b-4aa4-bf0f-8cf49f386dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:15.303512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d484e3a-0f6b-4aa4-bf0f-8cf49f386dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:15.303788 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d484e3a-0f6b-4aa4-bf0f-8cf49f386dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:15.304557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d484e3a-0f6b-4aa4-bf0f-8cf49f386dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:15.306123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d484e3a-0f6b-4aa4-bf0f-8cf49f386dc5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:15.317710 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1214/2472] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:45:15.322770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d484e3a-0f6b-4aa4-bf0f-8cf49f386dc5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:15.333124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d484e3a-0f6b-4aa4-bf0f-8cf49f386dc5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:15.338267 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd6079-5671-41ba-a7a1-346cb5bcc2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:15.338920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd6079-5671-41ba-a7a1-346cb5bcc2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:15.339194 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd6079-5671-41ba-a7a1-346cb5bcc2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:15.339455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd6079-5671-41ba-a7a1-346cb5bcc2dc admin admin] PATH_INFO: `/v3/projects/917dc238190c42ea88ead51c37a58268/users/04bfd320ceca446fb284aad3ba4bd6bc/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:15.341861 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd6079-5671-41ba-a7a1-346cb5bcc2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=917dc238190c42ea88ead51c37a58268, user_id=04bfd320ceca446fb284aad3ba4bd6bc, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:15.344648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd6079-5671-41ba-a7a1-346cb5bcc2dc admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:15.355566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1215/2473] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:45:15 2022] PUT /identity/v3/projects/917dc238190c42ea88ead51c37a58268/users/04bfd320ceca446fb284aad3ba4bd6bc/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:45:15.376039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bc918-c6d5-4fc2-9112-baa1be8ef6b2 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:15.376319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bc918-c6d5-4fc2-9112-baa1be8ef6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:15.376593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bc918-c6d5-4fc2-9112-baa1be8ef6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:15.408460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-051bc918-c6d5-4fc2-9112-baa1be8ef6b2 None None] MFA Rules not processed for user `04bfd320ceca446fb284aad3ba4bd6bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:45:15.409749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-051bc918-c6d5-4fc2-9112-baa1be8ef6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:15.489354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1216/2474] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:45:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:45:15.502859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bc918-c6d5-4fc2-9112-baa1be8ef6b2 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:15.505115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-051bc918-c6d5-4fc2-9112-baa1be8ef6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:15.535792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353863ab3a164b9fa49644358223096a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:15.568258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bc918-c6d5-4fc2-9112-baa1be8ef6b2 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:15.572076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551c6d6-c5a0-4418-8350-d230ee2e355f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:15.572727 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551c6d6-c5a0-4418-8350-d230ee2e355f service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:15.573241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551c6d6-c5a0-4418-8350-d230ee2e355f service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:15.573544 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551c6d6-c5a0-4418-8350-d230ee2e355f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:15.582893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551c6d6-c5a0-4418-8350-d230ee2e355f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:15.584398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551c6d6-c5a0-4418-8350-d230ee2e355f service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:15.594470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1217/2475] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:45:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:45:15.663023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353863ab3a164b9fa49644358223096a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:15.677057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353863ab3a164b9fa49644358223096a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:15.677732 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353863ab3a164b9fa49644358223096a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:15.684831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353863ab3a164b9fa49644358223096a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:15.691028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1259/2476] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:45:15 2022] DELETE /identity/v3/users/353863ab3a164b9fa49644358223096a => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:45:15.901107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551c6d6-c5a0-4418-8350-d230ee2e355f service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:15.910659 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551c6d6-c5a0-4418-8350-d230ee2e355f service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:15.961500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:15.962118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:15.962444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:15.962717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:15.964623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:16.026818 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:16.029891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:16.030039 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:16.030974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:16.045512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1218/2477] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:45:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:45:16.109830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f04c74-7880-4fde-bbe4-08b288c5134f admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:16.114490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:16.115115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:16.115394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:16.115665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] PATH_INFO: `/v3/projects/2157fe55e5744b6bbcb6afecbff2caf2` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:16.116963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2157fe55e5744b6bbcb6afecbff2caf2)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:16.118808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:16.194965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:16.289888 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2157fe55e5744b6bbcb6afecbff2caf2', 'request_id': 'req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:16.290943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:16.297810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2157fe55e5744b6bbcb6afecbff2caf2', 'request_id': 'req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:16.414727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2157fe55e5744b6bbcb6afecbff2caf2', 'request_id': 'req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:16.432089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] The token cache is being invalidate because project 2157fe55e5744b6bbcb6afecbff2caf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:45:16.432382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:16.433063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:45:16.435185 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1260/2478] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:45:16 2022] DELETE /identity/v3/projects/2157fe55e5744b6bbcb6afecbff2caf2 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:45:26.589548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=28 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:26.779168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=6,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:45:44.364685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:45:44.366371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:44.444311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae792a-e066-4858-ac2a-d64fe66b082f service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:45:44.507116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:45:44.507758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:45:44.508040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:45:44.508311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:45:44.510166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:45:44.521723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:45:44.523261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:45:44.533092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1219/2479] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:45:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:45:54.528893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=58 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:06.002606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:06.004905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:06.084277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37f9f7-60cd-4c31-b5e3-1bc5ef4dbdb6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:06.089253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc690-8226-437d-9d0e-ab8600956753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:06.089977 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc690-8226-437d-9d0e-ab8600956753 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:06.090302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc690-8226-437d-9d0e-ab8600956753 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:06.090615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fc690-8226-437d-9d0e-ab8600956753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:06.093300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f04fc690-8226-437d-9d0e-ab8600956753 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:06.168433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fc690-8226-437d-9d0e-ab8600956753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:06.169914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fc690-8226-437d-9d0e-ab8600956753 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:06.180636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1261/2480] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:11.163638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:11.165482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:11.180775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a50957-fda6-4af0-8be9-f84d54609a96 service placement] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:11.185540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:11.192178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:11.192178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:11.192178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] PATH_INFO: `/v3/users/4a80d0127ee349d69f0ed949bcb4c919` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:11.192178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a80d0127ee349d69f0ed949bcb4c919)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:11.192178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:11.470377 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a80d0127ee349d69f0ed949bcb4c919'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:11.470954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a80d0127ee349d69f0ed949bcb4c919'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:11.471577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a80d0127ee349d69f0ed949bcb4c919'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:11.478591 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a80d0127ee349d69f0ed949bcb4c919'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:11.491138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-9d64c35c-1360-4236-9c8f-5bf4fa232afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a80d0127ee349d69f0ed949bcb4c919'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:11.714587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1220/2481] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:11 2022] DELETE /identity/v3/users/4a80d0127ee349d69f0ed949bcb4c919 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:11.999528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc690-8226-437d-9d0e-ab8600956753 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:12.009385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fc690-8226-437d-9d0e-ab8600956753 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:12.071665 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:12.072457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:12.073349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:12.073620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] PATH_INFO: `/v3/projects/177e00cee82c44a196eaf5d0dc07b7f2` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:12.074862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=177e00cee82c44a196eaf5d0dc07b7f2)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:12.076767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:12.292029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '177e00cee82c44a196eaf5d0dc07b7f2', 'request_id': 'req-78aa3181-2ddc-4dc9-97f4-38130c573471'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:12.294047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:12.298596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '177e00cee82c44a196eaf5d0dc07b7f2', 'request_id': 'req-78aa3181-2ddc-4dc9-97f4-38130c573471'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:12.407172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '177e00cee82c44a196eaf5d0dc07b7f2', 'request_id': 'req-78aa3181-2ddc-4dc9-97f4-38130c573471'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:12.418803 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] The token cache is being invalidate because project 177e00cee82c44a196eaf5d0dc07b7f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:12.419297 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:12.420252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-78aa3181-2ddc-4dc9-97f4-38130c573471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:12.421786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1262/2482] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:11 2022] DELETE /identity/v3/projects/177e00cee82c44a196eaf5d0dc07b7f2 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:12.426769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a0843-2f26-4d59-9997-903a46b2b40c None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:12.427264 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a0843-2f26-4d59-9997-903a46b2b40c None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:12.427364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a0843-2f26-4d59-9997-903a46b2b40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:12.448984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-0a9a0843-2f26-4d59-9997-903a46b2b40c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:12.450238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0a9a0843-2f26-4d59-9997-903a46b2b40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:12.511968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1221/2483] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:12.519959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9ed04-1c10-4b13-9a54-d4296c5d96a4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:12.520240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9ed04-1c10-4b13-9a54-d4296c5d96a4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:12.520508 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9ed04-1c10-4b13-9a54-d4296c5d96a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:12.541054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2cd9ed04-1c10-4b13-9a54-d4296c5d96a4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:12.542388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2cd9ed04-1c10-4b13-9a54-d4296c5d96a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:12.550275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1263/2484] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:12.554550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a0843-2f26-4d59-9997-903a46b2b40c None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:12.564573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a0843-2f26-4d59-9997-903a46b2b40c None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:12.569149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34facfa-e9dd-4639-a3a8-eae4439808ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:12.569750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34facfa-e9dd-4639-a3a8-eae4439808ae admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:12.570028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34facfa-e9dd-4639-a3a8-eae4439808ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:12.570293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34facfa-e9dd-4639-a3a8-eae4439808ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:12.570926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34facfa-e9dd-4639-a3a8-eae4439808ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:12.571277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34facfa-e9dd-4639-a3a8-eae4439808ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:12.573715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34facfa-e9dd-4639-a3a8-eae4439808ae admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:12.585437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1222/2485] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:46:12.590509 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9ed04-1c10-4b13-9a54-d4296c5d96a4 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:12.599898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9ed04-1c10-4b13-9a54-d4296c5d96a4 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:12.604733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c8283-cd91-4f8a-97ab-3936fb725fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:12.605373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c8283-cd91-4f8a-97ab-3936fb725fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:12.605650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c8283-cd91-4f8a-97ab-3936fb725fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:12.605920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c8283-cd91-4f8a-97ab-3936fb725fa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:12.606543 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c8283-cd91-4f8a-97ab-3936fb725fa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:12.608448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c8283-cd91-4f8a-97ab-3936fb725fa1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:12.747419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1264/2486] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:12 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:12.748566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34facfa-e9dd-4639-a3a8-eae4439808ae admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:12.759463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34facfa-e9dd-4639-a3a8-eae4439808ae admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:12.814909 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a2ed5-71a9-4d45-a6ac-08e0359eaec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:12.815510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a2ed5-71a9-4d45-a6ac-08e0359eaec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:12.815786 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a2ed5-71a9-4d45-a6ac-08e0359eaec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:12.816057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a2ed5-71a9-4d45-a6ac-08e0359eaec0 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:12.816665 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a2ed5-71a9-4d45-a6ac-08e0359eaec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:12.818624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a2ed5-71a9-4d45-a6ac-08e0359eaec0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:12.826871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:12.959281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1223/2487] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:12 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:12.963627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c8283-cd91-4f8a-97ab-3936fb725fa1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:12.973978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c8283-cd91-4f8a-97ab-3936fb725fa1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:12.979087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbd85a-0ff5-473d-9c23-15a2a63ce828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:12.979785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbd85a-0ff5-473d-9c23-15a2a63ce828 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:12.980061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbd85a-0ff5-473d-9c23-15a2a63ce828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:12.980396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbd85a-0ff5-473d-9c23-15a2a63ce828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:12.981034 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbd85a-0ff5-473d-9c23-15a2a63ce828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:12.985577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbd85a-0ff5-473d-9c23-15a2a63ce828 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:12.993063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1265/2488] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:12.997811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a2ed5-71a9-4d45-a6ac-08e0359eaec0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:13.008195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a2ed5-71a9-4d45-a6ac-08e0359eaec0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:13.012917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edca850-941d-4ff9-a1b8-dc3b1e732bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:13.014142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edca850-941d-4ff9-a1b8-dc3b1e732bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:13.014142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edca850-941d-4ff9-a1b8-dc3b1e732bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:13.014142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edca850-941d-4ff9-a1b8-dc3b1e732bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:13.014638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edca850-941d-4ff9-a1b8-dc3b1e732bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:13.016246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edca850-941d-4ff9-a1b8-dc3b1e732bb8 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:13.026865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1224/2489] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:13.031335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbd85a-0ff5-473d-9c23-15a2a63ce828 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:13.042158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbd85a-0ff5-473d-9c23-15a2a63ce828 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:13.046905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6c33a-51b5-4a7d-b7ed-3b2be85a1267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:13.047539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6c33a-51b5-4a7d-b7ed-3b2be85a1267 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:13.047813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6c33a-51b5-4a7d-b7ed-3b2be85a1267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:13.048083 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6c33a-51b5-4a7d-b7ed-3b2be85a1267 admin admin] PATH_INFO: `/v3/projects/584a03e1f44840368199fdcb3f4c29da/users/41d978415a4146d3b63c9904e8cf6d8c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:13.063328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6c33a-51b5-4a7d-b7ed-3b2be85a1267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=584a03e1f44840368199fdcb3f4c29da, user_id=41d978415a4146d3b63c9904e8cf6d8c, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:13.066064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6c33a-51b5-4a7d-b7ed-3b2be85a1267 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:13.153446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1266/2490] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:13 2022] PUT /identity/v3/projects/584a03e1f44840368199fdcb3f4c29da/users/41d978415a4146d3b63c9904e8cf6d8c/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:13.165077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2b569-f206-43c3-a987-b825d272ac0d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:13.165442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2b569-f206-43c3-a987-b825d272ac0d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:13.165714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e2b569-f206-43c3-a987-b825d272ac0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:13.194148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-d8e2b569-f206-43c3-a987-b825d272ac0d None None] MFA Rules not processed for user `41d978415a4146d3b63c9904e8cf6d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:13.195347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d8e2b569-f206-43c3-a987-b825d272ac0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:13.264302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1225/2491] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:13.268647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6c33a-51b5-4a7d-b7ed-3b2be85a1267 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:13.333148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6c33a-51b5-4a7d-b7ed-3b2be85a1267 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:13.337895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ee9e0-dba3-4f1b-8c55-2dccd1ccc0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:13.339650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ee9e0-dba3-4f1b-8c55-2dccd1ccc0f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:13.340122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ee9e0-dba3-4f1b-8c55-2dccd1ccc0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:13.340682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ee9e0-dba3-4f1b-8c55-2dccd1ccc0f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:13.346144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ee9e0-dba3-4f1b-8c55-2dccd1ccc0f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:13.346144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ee9e0-dba3-4f1b-8c55-2dccd1ccc0f6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:13.480724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1267/2492] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:13 2022] POST /identity/v3/projects => generated 387 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:13.485044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2b569-f206-43c3-a987-b825d272ac0d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:13.496667 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e2b569-f206-43c3-a987-b825d272ac0d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:13.554441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24261774-5d5c-47c1-8815-1cfba5465b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:13.555038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24261774-5d5c-47c1-8815-1cfba5465b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:13.555311 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24261774-5d5c-47c1-8815-1cfba5465b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:13.555580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24261774-5d5c-47c1-8815-1cfba5465b05 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:13.556211 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24261774-5d5c-47c1-8815-1cfba5465b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:13.560143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24261774-5d5c-47c1-8815-1cfba5465b05 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:13.662918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1226/2493] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:13 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:13.667277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ee9e0-dba3-4f1b-8c55-2dccd1ccc0f6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:13.678451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ee9e0-dba3-4f1b-8c55-2dccd1ccc0f6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:13.683720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b6d51-690f-491c-b662-254c173518c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:13.684325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b6d51-690f-491c-b662-254c173518c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:13.684602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b6d51-690f-491c-b662-254c173518c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:13.684905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b6d51-690f-491c-b662-254c173518c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:13.685548 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b6d51-690f-491c-b662-254c173518c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:13.687433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b6d51-690f-491c-b662-254c173518c4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:13.699589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1268/2494] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:13.703670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24261774-5d5c-47c1-8815-1cfba5465b05 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:13.714295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24261774-5d5c-47c1-8815-1cfba5465b05 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:13.719367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26248b-2545-4537-9ad9-7829cb7769f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:13.719983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26248b-2545-4537-9ad9-7829cb7769f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:13.720261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26248b-2545-4537-9ad9-7829cb7769f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:13.720533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26248b-2545-4537-9ad9-7829cb7769f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:13.721204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26248b-2545-4537-9ad9-7829cb7769f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:13.722815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26248b-2545-4537-9ad9-7829cb7769f0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:13.734053 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1227/2495] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:13.738308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b6d51-690f-491c-b662-254c173518c4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:13.749157 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b6d51-690f-491c-b662-254c173518c4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:13.754111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6056d59-dd36-4895-ae7e-6213b57c7c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:13.754743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6056d59-dd36-4895-ae7e-6213b57c7c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:13.755012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6056d59-dd36-4895-ae7e-6213b57c7c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:13.755279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6056d59-dd36-4895-ae7e-6213b57c7c70 admin admin] PATH_INFO: `/v3/projects/93b9bd02ab13451b8135b5fd0603b4d0/users/74918d7253f5468ea42b47bfbe2ad3bc/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:13.769806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6056d59-dd36-4895-ae7e-6213b57c7c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b9bd02ab13451b8135b5fd0603b4d0, user_id=74918d7253f5468ea42b47bfbe2ad3bc, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:13.772415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6056d59-dd36-4895-ae7e-6213b57c7c70 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:13.840540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1269/2496] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:13 2022] PUT /identity/v3/projects/93b9bd02ab13451b8135b5fd0603b4d0/users/74918d7253f5468ea42b47bfbe2ad3bc/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:13.850860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50d69f-77b4-43d6-9926-f3adeb3c6bdd None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:13.851176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50d69f-77b4-43d6-9926-f3adeb3c6bdd None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:13.851452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50d69f-77b4-43d6-9926-f3adeb3c6bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:13.881797 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-eb50d69f-77b4-43d6-9926-f3adeb3c6bdd None None] MFA Rules not processed for user `74918d7253f5468ea42b47bfbe2ad3bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:13.882922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-eb50d69f-77b4-43d6-9926-f3adeb3c6bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:13.918929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:13.950137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1228/2497] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:13.954714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6056d59-dd36-4895-ae7e-6213b57c7c70 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:13.985560 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:14.020925 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6056d59-dd36-4895-ae7e-6213b57c7c70 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:14.027257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358bfcd-4771-4946-a531-da0eb3d45b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:14.028018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358bfcd-4771-4946-a531-da0eb3d45b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:14.028354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358bfcd-4771-4946-a531-da0eb3d45b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:14.028703 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358bfcd-4771-4946-a531-da0eb3d45b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:14.029359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358bfcd-4771-4946-a531-da0eb3d45b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:14.031263 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358bfcd-4771-4946-a531-da0eb3d45b31 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:14.123759 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1270/2498] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:13 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:14.128058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50d69f-77b4-43d6-9926-f3adeb3c6bdd None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:14.140146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50d69f-77b4-43d6-9926-f3adeb3c6bdd None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:14.196705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019e343-a078-4aa4-896e-b43e3384cebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:14.198165 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019e343-a078-4aa4-896e-b43e3384cebf admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:14.198455 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019e343-a078-4aa4-896e-b43e3384cebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:14.198883 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019e343-a078-4aa4-896e-b43e3384cebf admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:14.199554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019e343-a078-4aa4-896e-b43e3384cebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:14.201508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019e343-a078-4aa4-896e-b43e3384cebf admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:14.294023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1229/2499] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:14 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:14.298068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358bfcd-4771-4946-a531-da0eb3d45b31 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:14.310123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358bfcd-4771-4946-a531-da0eb3d45b31 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:14.314770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3abf1-ffb7-4219-9456-0d62c6cc9fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:14.315380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3abf1-ffb7-4219-9456-0d62c6cc9fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:14.315660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3abf1-ffb7-4219-9456-0d62c6cc9fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:14.315935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3abf1-ffb7-4219-9456-0d62c6cc9fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:14.316574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3abf1-ffb7-4219-9456-0d62c6cc9fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:14.318276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3abf1-ffb7-4219-9456-0d62c6cc9fe1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:14.330705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1271/2500] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:14.335012 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019e343-a078-4aa4-896e-b43e3384cebf admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:14.346079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019e343-a078-4aa4-896e-b43e3384cebf admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:14.351116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6a4af-1b6d-4520-99cb-60c6e97bd948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:14.351777 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6a4af-1b6d-4520-99cb-60c6e97bd948 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:14.352057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6a4af-1b6d-4520-99cb-60c6e97bd948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:14.352338 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6a4af-1b6d-4520-99cb-60c6e97bd948 admin admin] PATH_INFO: `/v3/projects/53a06391f9874dc6b042bbf5b7e4d6c1/users/515b99f46cd34acca631bc75c091a706/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:14.368694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6a4af-1b6d-4520-99cb-60c6e97bd948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a06391f9874dc6b042bbf5b7e4d6c1, user_id=515b99f46cd34acca631bc75c091a706, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:14.372646 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6a4af-1b6d-4520-99cb-60c6e97bd948 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:14.487463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1230/2501] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:14 2022] PUT /identity/v3/projects/53a06391f9874dc6b042bbf5b7e4d6c1/users/515b99f46cd34acca631bc75c091a706/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:14.491546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3abf1-ffb7-4219-9456-0d62c6cc9fe1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:14.502725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3abf1-ffb7-4219-9456-0d62c6cc9fe1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:14.562913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ed4ab-920c-4831-ad25-8a071c2137df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:14.563528 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ed4ab-920c-4831-ad25-8a071c2137df admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:14.563808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ed4ab-920c-4831-ad25-8a071c2137df admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:14.564079 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ed4ab-920c-4831-ad25-8a071c2137df admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:14.564719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ed4ab-920c-4831-ad25-8a071c2137df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:14.566342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ed4ab-920c-4831-ad25-8a071c2137df admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:14.577110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1272/2502] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:14.581473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6a4af-1b6d-4520-99cb-60c6e97bd948 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:14.591758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6a4af-1b6d-4520-99cb-60c6e97bd948 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:14.602290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf16af0-1b44-4838-890d-d50693301a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:14.602956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf16af0-1b44-4838-890d-d50693301a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:14.603231 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf16af0-1b44-4838-890d-d50693301a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:14.604053 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf16af0-1b44-4838-890d-d50693301a74 admin admin] PATH_INFO: `/v3/projects/53a06391f9874dc6b042bbf5b7e4d6c1/users/515b99f46cd34acca631bc75c091a706/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:14.606253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf16af0-1b44-4838-890d-d50693301a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a06391f9874dc6b042bbf5b7e4d6c1, user_id=515b99f46cd34acca631bc75c091a706, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:14.609080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf16af0-1b44-4838-890d-d50693301a74 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:14.620656 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1231/2503] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:14 2022] PUT /identity/v3/projects/53a06391f9874dc6b042bbf5b7e4d6c1/users/515b99f46cd34acca631bc75c091a706/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:14.632654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5aff3-02d3-4ab6-bc44-620bf41ac0f9 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:14.632971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5aff3-02d3-4ab6-bc44-620bf41ac0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:14.633343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5aff3-02d3-4ab6-bc44-620bf41ac0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:14.662367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-15a5aff3-02d3-4ab6-bc44-620bf41ac0f9 None None] MFA Rules not processed for user `515b99f46cd34acca631bc75c091a706`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:14.663573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-15a5aff3-02d3-4ab6-bc44-620bf41ac0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:14.734968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1273/2504] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:14.746205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf16af0-1b44-4838-890d-d50693301a74 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:14.747945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ecf16af0-1b44-4838-890d-d50693301a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:14.806390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf16af0-1b44-4838-890d-d50693301a74 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:14.810633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644925cd-1b54-428f-86d7-12cdcaae35cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:14.811240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644925cd-1b54-428f-86d7-12cdcaae35cf service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:14.811510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644925cd-1b54-428f-86d7-12cdcaae35cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:14.811782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644925cd-1b54-428f-86d7-12cdcaae35cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:14.821178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644925cd-1b54-428f-86d7-12cdcaae35cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:14.823154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644925cd-1b54-428f-86d7-12cdcaae35cf service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:14.833020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1232/2505] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:15.897567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5aff3-02d3-4ab6-bc44-620bf41ac0f9 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:15.898959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-15a5aff3-02d3-4ab6-bc44-620bf41ac0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:15.912887 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5aff3-02d3-4ab6-bc44-620bf41ac0f9 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:15.916749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ace679-d5b2-46ea-9f51-e6d30cfe0386 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:15.917388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ace679-d5b2-46ea-9f51-e6d30cfe0386 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:15.917718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ace679-d5b2-46ea-9f51-e6d30cfe0386 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:15.917989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ace679-d5b2-46ea-9f51-e6d30cfe0386 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:15.938265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:15.976965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ace679-d5b2-46ea-9f51-e6d30cfe0386 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:15.979180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ace679-d5b2-46ea-9f51-e6d30cfe0386 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:15.988869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1274/2506] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:20.071770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644925cd-1b54-428f-86d7-12cdcaae35cf service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:20.073437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-644925cd-1b54-428f-86d7-12cdcaae35cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:20.115173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:20.142247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644925cd-1b54-428f-86d7-12cdcaae35cf service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:20.146077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e327f-4fce-4158-8120-0a36b80c48a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:20.146680 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e327f-4fce-4158-8120-0a36b80c48a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:20.146957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e327f-4fce-4158-8120-0a36b80c48a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:20.147227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e327f-4fce-4158-8120-0a36b80c48a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:20.149010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a16e327f-4fce-4158-8120-0a36b80c48a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:20.222874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e327f-4fce-4158-8120-0a36b80c48a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:20.224242 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e327f-4fce-4158-8120-0a36b80c48a3 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:20.235565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1233/2507] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:22.702360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ace679-d5b2-46ea-9f51-e6d30cfe0386 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:22.703863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-45ace679-d5b2-46ea-9f51-e6d30cfe0386 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:22.718226 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ace679-d5b2-46ea-9f51-e6d30cfe0386 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:22.722993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:22.723644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:22.723945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:22.725530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] PATH_INFO: `/v3/users/9d9823f2dd034a2bbb21b3f41a4292ea` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:22.725530 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9823f2dd034a2bbb21b3f41a4292ea)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:22.727349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:22.981849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9823f2dd034a2bbb21b3f41a4292ea'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:23.046906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9823f2dd034a2bbb21b3f41a4292ea'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:23.064731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9823f2dd034a2bbb21b3f41a4292ea'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:23.065491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9823f2dd034a2bbb21b3f41a4292ea'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:23.073327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9823f2dd034a2bbb21b3f41a4292ea'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:23.080551 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1275/2508] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:22 2022] DELETE /identity/v3/users/9d9823f2dd034a2bbb21b3f41a4292ea => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:23.085282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05621ec7-bb9b-4f0b-85a1-a3f6789c28b3 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:23.085739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05621ec7-bb9b-4f0b-85a1-a3f6789c28b3 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:23.085912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05621ec7-bb9b-4f0b-85a1-a3f6789c28b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:23.108188 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-05621ec7-bb9b-4f0b-85a1-a3f6789c28b3 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:23.111149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-05621ec7-bb9b-4f0b-85a1-a3f6789c28b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:23.182206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1234/2509] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:23.191842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:23.194753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:23.262733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d64687-9e22-4f3d-a25e-2e4e28dba862 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:23.266577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61579e06-a3e8-4cdd-bd17-31dcba2f4528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:23.267098 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61579e06-a3e8-4cdd-bd17-31dcba2f4528 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:23.267370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61579e06-a3e8-4cdd-bd17-31dcba2f4528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:23.267640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61579e06-a3e8-4cdd-bd17-31dcba2f4528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:23.279044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61579e06-a3e8-4cdd-bd17-31dcba2f4528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:23.280583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61579e06-a3e8-4cdd-bd17-31dcba2f4528 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:23.292583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1276/2510] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:23.649516 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05621ec7-bb9b-4f0b-85a1-a3f6789c28b3 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:23.653036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-05621ec7-bb9b-4f0b-85a1-a3f6789c28b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:23.753391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05621ec7-bb9b-4f0b-85a1-a3f6789c28b3 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:23.763550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269bba1-6562-4603-b90d-da688882130a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:23.764281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269bba1-6562-4603-b90d-da688882130a service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:23.764489 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269bba1-6562-4603-b90d-da688882130a service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:23.764901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269bba1-6562-4603-b90d-da688882130a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:23.767528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f269bba1-6562-4603-b90d-da688882130a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:23.837355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:23.868511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61579e06-a3e8-4cdd-bd17-31dcba2f4528 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:23.871439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-61579e06-a3e8-4cdd-bd17-31dcba2f4528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:23.880968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269bba1-6562-4603-b90d-da688882130a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:23.880968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f269bba1-6562-4603-b90d-da688882130a service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:23.890550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61579e06-a3e8-4cdd-bd17-31dcba2f4528 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:23.896324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1235/2511] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:23.925645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:23.925645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:23.925645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:23.925645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] PATH_INFO: `/v3/projects/00a60cf91e794dcba7c9ef68349b22f0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:23.927171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a60cf91e794dcba7c9ef68349b22f0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:23.929393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:24.081681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:24.095206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a60cf91e794dcba7c9ef68349b22f0', 'request_id': 'req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.101458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a60cf91e794dcba7c9ef68349b22f0', 'request_id': 'req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.253558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a60cf91e794dcba7c9ef68349b22f0', 'request_id': 'req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.265533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] The token cache is being invalidate because project 00a60cf91e794dcba7c9ef68349b22f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:24.265962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.266606 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.268113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1277/2512] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:23 2022] DELETE /identity/v3/projects/00a60cf91e794dcba7c9ef68349b22f0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:24.272141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269bba1-6562-4603-b90d-da688882130a service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:24.285096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f269bba1-6562-4603-b90d-da688882130a service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:24.348685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:24.349528 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:24.349813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:24.350110 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] PATH_INFO: `/v3/users/04bfd320ceca446fb284aad3ba4bd6bc` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:24.351274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=04bfd320ceca446fb284aad3ba4bd6bc)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:24.353912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:24.400472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:24.401913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:24.475786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11054c6e-d7d0-41a3-b3f0-ca7d8c922534 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:24.482561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:24.483204 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:24.483456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:24.483731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:24.485544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:24.498643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:24.500476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:24.513430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1278/2513] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:24.561858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:24.563302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:24.578696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfe64a-8b93-4b03-bb0f-731374e13cb6 service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:24.583607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:24.584288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:24.584514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:24.584786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] PATH_INFO: `/v3/users/2715cf2a5129439c83a5347bb963f1a9` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:24.585939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2715cf2a5129439c83a5347bb963f1a9)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:24.588519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:24.675894 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04bfd320ceca446fb284aad3ba4bd6bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.676538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04bfd320ceca446fb284aad3ba4bd6bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.677204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04bfd320ceca446fb284aad3ba4bd6bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.684785 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04bfd320ceca446fb284aad3ba4bd6bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.696423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04bfd320ceca446fb284aad3ba4bd6bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:24.854028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1236/2514] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:24 2022] DELETE /identity/v3/users/04bfd320ceca446fb284aad3ba4bd6bc => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:24.861727 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2715cf2a5129439c83a5347bb963f1a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.038797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2715cf2a5129439c83a5347bb963f1a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.052558 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2715cf2a5129439c83a5347bb963f1a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.053306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2715cf2a5129439c83a5347bb963f1a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.060664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2715cf2a5129439c83a5347bb963f1a9'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.066923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1279/2515] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:24 2022] DELETE /identity/v3/users/2715cf2a5129439c83a5347bb963f1a9 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:25.395987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:25.398538 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:25.430539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:25.433692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:25.471776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:25.488028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:25.494287 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d869a-316a-4cb4-a904-98ca903593bb admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:25.529571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ae027-9102-4003-aee1-d27bc7adcfe2 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:25.534225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:25.534849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:25.535122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:25.535390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] PATH_INFO: `/v3/projects/4a4159dc366f41bc9e4057a079f6fb5a` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:25.536483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4159dc366f41bc9e4057a079f6fb5a)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:25.538718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:25.555173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:25.555906 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:25.556254 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:25.557141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] PATH_INFO: `/v3/projects/917dc238190c42ea88ead51c37a58268` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:25.558332 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=917dc238190c42ea88ead51c37a58268)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:25.560235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:25.699948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4159dc366f41bc9e4057a079f6fb5a', 'request_id': 'req-514d4b90-eae3-425c-893d-7671d8fb9eb3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.709517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4159dc366f41bc9e4057a079f6fb5a', 'request_id': 'req-514d4b90-eae3-425c-893d-7671d8fb9eb3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.726356 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917dc238190c42ea88ead51c37a58268', 'request_id': 'req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.736872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917dc238190c42ea88ead51c37a58268', 'request_id': 'req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.870743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4159dc366f41bc9e4057a079f6fb5a', 'request_id': 'req-514d4b90-eae3-425c-893d-7671d8fb9eb3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.881985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] The token cache is being invalidate because project 4a4159dc366f41bc9e4057a079f6fb5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:25.882275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.883967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-514d4b90-eae3-425c-893d-7671d8fb9eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.885740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1280/2516] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:25 2022] DELETE /identity/v3/projects/4a4159dc366f41bc9e4057a079f6fb5a => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:25.890608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54208770-db8d-4ef2-9441-5bb333f3b77a None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:25.890827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54208770-db8d-4ef2-9441-5bb333f3b77a None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:25.891094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54208770-db8d-4ef2-9441-5bb333f3b77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:25.911550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-54208770-db8d-4ef2-9441-5bb333f3b77a None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:25.912786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-54208770-db8d-4ef2-9441-5bb333f3b77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:25.947872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917dc238190c42ea88ead51c37a58268', 'request_id': 'req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.961788 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] The token cache is being invalidate because project 917dc238190c42ea88ead51c37a58268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:25.962082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.962918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-71bbf1b1-0477-4d89-b11a-1689a07b7ee9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:25.964767 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1237/2517] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:25 2022] DELETE /identity/v3/projects/917dc238190c42ea88ead51c37a58268 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:25.969397 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414300fa-7b5d-44e1-8a8a-62bc5ed64a06 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:25.969768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414300fa-7b5d-44e1-8a8a-62bc5ed64a06 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:25.970213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414300fa-7b5d-44e1-8a8a-62bc5ed64a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:25.977057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1281/2518] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:25.985629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92dbe-b903-4aad-914b-de69ce9882d7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:25.985965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92dbe-b903-4aad-914b-de69ce9882d7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:25.986201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92dbe-b903-4aad-914b-de69ce9882d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:25.998207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-414300fa-7b5d-44e1-8a8a-62bc5ed64a06 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:25.999558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-414300fa-7b5d-44e1-8a8a-62bc5ed64a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:26.007890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4cd92dbe-b903-4aad-914b-de69ce9882d7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:26.009273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4cd92dbe-b903-4aad-914b-de69ce9882d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:26.014619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1238/2519] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:26.021174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1282/2520] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:26.021970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fb23b-4d39-4f59-a475-611a0ad49257 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.022182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd92dbe-b903-4aad-914b-de69ce9882d7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.022290 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fb23b-4d39-4f59-a475-611a0ad49257 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.022512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fb23b-4d39-4f59-a475-611a0ad49257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.032303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd92dbe-b903-4aad-914b-de69ce9882d7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.037826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1cdb1-4964-4e11-9613-11b670df35ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.038436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1cdb1-4964-4e11-9613-11b670df35ce admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.038740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1cdb1-4964-4e11-9613-11b670df35ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.039015 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff1cdb1-4964-4e11-9613-11b670df35ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.039837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1cdb1-4964-4e11-9613-11b670df35ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:26.040326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1cdb1-4964-4e11-9613-11b670df35ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.042454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff1cdb1-4964-4e11-9613-11b670df35ce admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.047437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cf8fb23b-4d39-4f59-a475-611a0ad49257 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:26.048764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cf8fb23b-4d39-4f59-a475-611a0ad49257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:26.055356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1283/2521] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:46:26.060306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1cdb1-4964-4e11-9613-11b670df35ce admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.070422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1239/2522] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:26.070918 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff1cdb1-4964-4e11-9613-11b670df35ce admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.071462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fb23b-4d39-4f59-a475-611a0ad49257 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.079737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cce431-d4ec-4a26-ae4d-fb92e5ee7960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.080343 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cce431-d4ec-4a26-ae4d-fb92e5ee7960 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.080615 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cce431-d4ec-4a26-ae4d-fb92e5ee7960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.080921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cce431-d4ec-4a26-ae4d-fb92e5ee7960 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.081654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cce431-d4ec-4a26-ae4d-fb92e5ee7960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.082956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fb23b-4d39-4f59-a475-611a0ad49257 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.083520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cce431-d4ec-4a26-ae4d-fb92e5ee7960 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.091073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287cabd-d339-408f-885f-2d060b82353f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.095813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287cabd-d339-408f-885f-2d060b82353f admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.095813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287cabd-d339-408f-885f-2d060b82353f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.095813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287cabd-d339-408f-885f-2d060b82353f admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.095813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287cabd-d339-408f-885f-2d060b82353f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:26.095813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287cabd-d339-408f-885f-2d060b82353f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.095813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287cabd-d339-408f-885f-2d060b82353f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.112451 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1240/2523] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:46:26.113541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287cabd-d339-408f-885f-2d060b82353f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.124621 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287cabd-d339-408f-885f-2d060b82353f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.129495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad3e2c-b405-4dcf-bf6c-2881d78bbbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.130080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad3e2c-b405-4dcf-bf6c-2881d78bbbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.130359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad3e2c-b405-4dcf-bf6c-2881d78bbbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.130630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad3e2c-b405-4dcf-bf6c-2881d78bbbb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.131339 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad3e2c-b405-4dcf-bf6c-2881d78bbbb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.133246 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad3e2c-b405-4dcf-bf6c-2881d78bbbb8 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.207222 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1284/2524] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:26 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:26.211245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cce431-d4ec-4a26-ae4d-fb92e5ee7960 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.221293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:26.273822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1241/2525] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:26 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:26.276432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cce431-d4ec-4a26-ae4d-fb92e5ee7960 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.279683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad3e2c-b405-4dcf-bf6c-2881d78bbbb8 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.331633 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d8877-3815-41bb-b816-be20e474bf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.332244 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d8877-3815-41bb-b816-be20e474bf64 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.332524 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d8877-3815-41bb-b816-be20e474bf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.332812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d8877-3815-41bb-b816-be20e474bf64 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.333442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d8877-3815-41bb-b816-be20e474bf64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.335372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d8877-3815-41bb-b816-be20e474bf64 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.340444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad3e2c-b405-4dcf-bf6c-2881d78bbbb8 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.345347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e799771-12eb-438e-be0f-9d14f3971949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.345952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e799771-12eb-438e-be0f-9d14f3971949 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.346229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e799771-12eb-438e-be0f-9d14f3971949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.346518 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e799771-12eb-438e-be0f-9d14f3971949 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.347132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e799771-12eb-438e-be0f-9d14f3971949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.349065 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e799771-12eb-438e-be0f-9d14f3971949 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.378423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:26.510510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1285/2526] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:26 2022] POST /identity/v3/users => generated 287 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:26.514546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d8877-3815-41bb-b816-be20e474bf64 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.524176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d8877-3815-41bb-b816-be20e474bf64 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.529547 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc32e95-fc26-4ee2-8117-8bc5034216e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.530131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc32e95-fc26-4ee2-8117-8bc5034216e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.530407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc32e95-fc26-4ee2-8117-8bc5034216e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.530673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc32e95-fc26-4ee2-8117-8bc5034216e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.531299 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc32e95-fc26-4ee2-8117-8bc5034216e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.532898 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc32e95-fc26-4ee2-8117-8bc5034216e7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.543120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1286/2527] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:26.546980 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc32e95-fc26-4ee2-8117-8bc5034216e7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.556363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc32e95-fc26-4ee2-8117-8bc5034216e7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.561003 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863e4eb-7aa7-4e4a-8b75-6d199974fea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.561569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863e4eb-7aa7-4e4a-8b75-6d199974fea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.561841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863e4eb-7aa7-4e4a-8b75-6d199974fea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.562111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863e4eb-7aa7-4e4a-8b75-6d199974fea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.562729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863e4eb-7aa7-4e4a-8b75-6d199974fea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.564273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863e4eb-7aa7-4e4a-8b75-6d199974fea0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.575793 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1287/2528] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:26.579789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863e4eb-7aa7-4e4a-8b75-6d199974fea0 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.588930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1242/2529] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:26 2022] POST /identity/v3/users => generated 283 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:26.592067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863e4eb-7aa7-4e4a-8b75-6d199974fea0 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.592596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e799771-12eb-438e-be0f-9d14f3971949 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.597086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a0568a-80e1-47f3-94ef-cf044e094c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.597877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0568a-80e1-47f3-94ef-cf044e094c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.598127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0568a-80e1-47f3-94ef-cf044e094c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.598875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0568a-80e1-47f3-94ef-cf044e094c7f admin admin] PATH_INFO: `/v3/projects/285bd7824c8542b7b57fa485e6923b29/users/c587b84384304a5689d1f3f8966bd9ce/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.602576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e799771-12eb-438e-be0f-9d14f3971949 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.607965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e46f5c-4ee5-4283-81e6-7e46d420b6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.608938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e46f5c-4ee5-4283-81e6-7e46d420b6da admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.609209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e46f5c-4ee5-4283-81e6-7e46d420b6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.609501 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e46f5c-4ee5-4283-81e6-7e46d420b6da admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.610076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e46f5c-4ee5-4283-81e6-7e46d420b6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.611846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e46f5c-4ee5-4283-81e6-7e46d420b6da admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.614848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0568a-80e1-47f3-94ef-cf044e094c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=285bd7824c8542b7b57fa485e6923b29, user_id=c587b84384304a5689d1f3f8966bd9ce, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.617585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a0568a-80e1-47f3-94ef-cf044e094c7f admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.622442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1243/2530] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:26.627061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e46f5c-4ee5-4283-81e6-7e46d420b6da admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.639234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e46f5c-4ee5-4283-81e6-7e46d420b6da admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.644203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5eff-299e-401b-8e8a-b1fc04514564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.644813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5eff-299e-401b-8e8a-b1fc04514564 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.645106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5eff-299e-401b-8e8a-b1fc04514564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.645431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5eff-299e-401b-8e8a-b1fc04514564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.646123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca5eff-299e-401b-8e8a-b1fc04514564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.647746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca5eff-299e-401b-8e8a-b1fc04514564 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.660791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1244/2531] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:26.665197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5eff-299e-401b-8e8a-b1fc04514564 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.675926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5eff-299e-401b-8e8a-b1fc04514564 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.680620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e0c33-6d3a-4500-af8f-a831022f8729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.681422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e0c33-6d3a-4500-af8f-a831022f8729 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.681853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e0c33-6d3a-4500-af8f-a831022f8729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.682136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e0c33-6d3a-4500-af8f-a831022f8729 admin admin] PATH_INFO: `/v3/projects/4624cfd084b543b592d34f7064cc3acb/users/d6c0d6aa73754070b951b9c9f0294682/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.698898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e0c33-6d3a-4500-af8f-a831022f8729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4624cfd084b543b592d34f7064cc3acb, user_id=d6c0d6aa73754070b951b9c9f0294682, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:26.701778 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e0c33-6d3a-4500-af8f-a831022f8729 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:26.738781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1288/2532] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:26 2022] PUT /identity/v3/projects/285bd7824c8542b7b57fa485e6923b29/users/c587b84384304a5689d1f3f8966bd9ce/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:26.752071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a68a0f-033d-4d60-8591-d5501c4d8085 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.752071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a68a0f-033d-4d60-8591-d5501c4d8085 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.752419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a68a0f-033d-4d60-8591-d5501c4d8085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.785723 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-85a68a0f-033d-4d60-8591-d5501c4d8085 None None] MFA Rules not processed for user `c587b84384304a5689d1f3f8966bd9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:26.786823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-85a68a0f-033d-4d60-8591-d5501c4d8085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:26.819071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1245/2533] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:26 2022] PUT /identity/v3/projects/4624cfd084b543b592d34f7064cc3acb/users/d6c0d6aa73754070b951b9c9f0294682/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:26.834770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475b280-da90-43bf-bdbc-6dac21da8e35 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.835202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475b280-da90-43bf-bdbc-6dac21da8e35 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.835616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475b280-da90-43bf-bdbc-6dac21da8e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.862415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1289/2534] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:26.865319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-c475b280-da90-43bf-bdbc-6dac21da8e35 None None] MFA Rules not processed for user `d6c0d6aa73754070b951b9c9f0294682`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:26.866864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a68a0f-033d-4d60-8591-d5501c4d8085 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.866997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c475b280-da90-43bf-bdbc-6dac21da8e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:26.933369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a68a0f-033d-4d60-8591-d5501c4d8085 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:26.936522 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1246/2535] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:26.941003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475b280-da90-43bf-bdbc-6dac21da8e35 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:26.986192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:26.997308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeb9a6-1911-4e2a-8a6d-e190a4441693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:26.997924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeb9a6-1911-4e2a-8a6d-e190a4441693 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:26.998198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeb9a6-1911-4e2a-8a6d-e190a4441693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:26.998471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeb9a6-1911-4e2a-8a6d-e190a4441693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:26.999182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeb9a6-1911-4e2a-8a6d-e190a4441693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.001347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeb9a6-1911-4e2a-8a6d-e190a4441693 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.016904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475b280-da90-43bf-bdbc-6dac21da8e35 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.021729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814b11-b21b-46a6-9eb3-e8232f9da925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.022383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814b11-b21b-46a6-9eb3-e8232f9da925 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.022658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814b11-b21b-46a6-9eb3-e8232f9da925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.022927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814b11-b21b-46a6-9eb3-e8232f9da925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.023542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82814b11-b21b-46a6-9eb3-e8232f9da925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.025823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82814b11-b21b-46a6-9eb3-e8232f9da925 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.130568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1290/2536] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:26 2022] POST /identity/v3/projects => generated 371 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:27.131103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1247/2537] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:26 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:27.131705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeb9a6-1911-4e2a-8a6d-e190a4441693 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.131820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814b11-b21b-46a6-9eb3-e8232f9da925 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.137036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:27.194744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814b11-b21b-46a6-9eb3-e8232f9da925 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.196824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeb9a6-1911-4e2a-8a6d-e190a4441693 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.200462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075b48-3c88-4f4e-9657-7936650eab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.201100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075b48-3c88-4f4e-9657-7936650eab49 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.201429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075b48-3c88-4f4e-9657-7936650eab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.201702 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05075b48-3c88-4f4e-9657-7936650eab49 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.202316 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05075b48-3c88-4f4e-9657-7936650eab49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.203102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad80b2d-c6cc-49a6-987e-587cf0dae047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.203710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad80b2d-c6cc-49a6-987e-587cf0dae047 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.204016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad80b2d-c6cc-49a6-987e-587cf0dae047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.204308 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05075b48-3c88-4f4e-9657-7936650eab49 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.204414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad80b2d-c6cc-49a6-987e-587cf0dae047 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.204979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad80b2d-c6cc-49a6-987e-587cf0dae047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.207081 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad80b2d-c6cc-49a6-987e-587cf0dae047 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.284572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1248/2538] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:27 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:27.285746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075b48-3c88-4f4e-9657-7936650eab49 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.296055 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05075b48-3c88-4f4e-9657-7936650eab49 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.300635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6efe8-492b-4556-973f-a4a7e256419c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.301256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6efe8-492b-4556-973f-a4a7e256419c admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.301536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6efe8-492b-4556-973f-a4a7e256419c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.301804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6efe8-492b-4556-973f-a4a7e256419c admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.302434 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6efe8-492b-4556-973f-a4a7e256419c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.304134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6efe8-492b-4556-973f-a4a7e256419c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.312426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1291/2539] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:27 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:27.316291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1249/2540] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:27.319427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad80b2d-c6cc-49a6-987e-587cf0dae047 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.319954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6efe8-492b-4556-973f-a4a7e256419c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.329731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad80b2d-c6cc-49a6-987e-587cf0dae047 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.330082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6efe8-492b-4556-973f-a4a7e256419c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.334808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4fc01-7aae-4689-82c5-6de08401337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.334992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845a753-445c-462f-afc5-2df7bc36c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.335452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4fc01-7aae-4689-82c5-6de08401337a admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.335559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845a753-445c-462f-afc5-2df7bc36c318 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.335728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4fc01-7aae-4689-82c5-6de08401337a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.335836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845a753-445c-462f-afc5-2df7bc36c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.335998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4fc01-7aae-4689-82c5-6de08401337a admin admin] PATH_INFO: `/v3/projects/e598273e91e949fc8dea71f562dffa83/users/949029f87d404d8bb790c56966cf99d1/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.336099 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845a753-445c-462f-afc5-2df7bc36c318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.336644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845a753-445c-462f-afc5-2df7bc36c318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.338578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845a753-445c-462f-afc5-2df7bc36c318 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.344212 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:27.349710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1292/2541] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:27.351275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4fc01-7aae-4689-82c5-6de08401337a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e598273e91e949fc8dea71f562dffa83, user_id=949029f87d404d8bb790c56966cf99d1, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.354450 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845a753-445c-462f-afc5-2df7bc36c318 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.354970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4fc01-7aae-4689-82c5-6de08401337a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.364755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845a753-445c-462f-afc5-2df7bc36c318 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.370199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b3d3c-ecdd-4811-9fa3-2123a9eb497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.370853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b3d3c-ecdd-4811-9fa3-2123a9eb497a admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.371224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b3d3c-ecdd-4811-9fa3-2123a9eb497a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.371496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b3d3c-ecdd-4811-9fa3-2123a9eb497a admin admin] PATH_INFO: `/v3/projects/e77cec56c8bc4e488b0ea8ab2e6b2e14/users/c671e5cd1b6e40e5b760c7def8844f58/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.386691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b3d3c-ecdd-4811-9fa3-2123a9eb497a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77cec56c8bc4e488b0ea8ab2e6b2e14, user_id=c671e5cd1b6e40e5b760c7def8844f58, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.389385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b3d3c-ecdd-4811-9fa3-2123a9eb497a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.464865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1250/2542] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:27 2022] PUT /identity/v3/projects/e598273e91e949fc8dea71f562dffa83/users/949029f87d404d8bb790c56966cf99d1/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:27.468466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4fc01-7aae-4689-82c5-6de08401337a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.503771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1293/2543] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:27 2022] PUT /identity/v3/projects/e77cec56c8bc4e488b0ea8ab2e6b2e14/users/c671e5cd1b6e40e5b760c7def8844f58/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:27.507140 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b3d3c-ecdd-4811-9fa3-2123a9eb497a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.546993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4fc01-7aae-4689-82c5-6de08401337a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.582909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b3d3c-ecdd-4811-9fa3-2123a9eb497a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.587919 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fbfc-335c-4081-879b-e0f00db98f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.588753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fbfc-335c-4081-879b-e0f00db98f5a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.589064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fbfc-335c-4081-879b-e0f00db98f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.589338 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017fbfc-335c-4081-879b-e0f00db98f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.589981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017fbfc-335c-4081-879b-e0f00db98f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.591637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017fbfc-335c-4081-879b-e0f00db98f5a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.604166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1294/2544] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:27.608229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fbfc-335c-4081-879b-e0f00db98f5a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.618731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330b99a-27ca-49dd-87bf-06b6f89be10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.620233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017fbfc-335c-4081-879b-e0f00db98f5a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.620390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330b99a-27ca-49dd-87bf-06b6f89be10f admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.620584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330b99a-27ca-49dd-87bf-06b6f89be10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.620934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330b99a-27ca-49dd-87bf-06b6f89be10f admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.621573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330b99a-27ca-49dd-87bf-06b6f89be10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.623166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330b99a-27ca-49dd-87bf-06b6f89be10f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.626795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3d57a-fbf4-404d-ad37-72ddf9cd5d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.627453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3d57a-fbf4-404d-ad37-72ddf9cd5d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.627732 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3d57a-fbf4-404d-ad37-72ddf9cd5d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.628005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3d57a-fbf4-404d-ad37-72ddf9cd5d62 admin admin] PATH_INFO: `/v3/projects/e77cec56c8bc4e488b0ea8ab2e6b2e14/users/c671e5cd1b6e40e5b760c7def8844f58/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.629967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3d57a-fbf4-404d-ad37-72ddf9cd5d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77cec56c8bc4e488b0ea8ab2e6b2e14, user_id=c671e5cd1b6e40e5b760c7def8844f58, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.631138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:27.632849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3d57a-fbf4-404d-ad37-72ddf9cd5d62 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.641732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1251/2545] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:27.644833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330b99a-27ca-49dd-87bf-06b6f89be10f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.647172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1295/2546] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:27 2022] PUT /identity/v3/projects/e77cec56c8bc4e488b0ea8ab2e6b2e14/users/c671e5cd1b6e40e5b760c7def8844f58/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:27.655938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330b99a-27ca-49dd-87bf-06b6f89be10f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.661312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550d72-4abe-4eba-9c54-3a31663dab9c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.661590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550d72-4abe-4eba-9c54-3a31663dab9c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.661868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550d72-4abe-4eba-9c54-3a31663dab9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.692880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6e550d72-4abe-4eba-9c54-3a31663dab9c None None] MFA Rules not processed for user `c671e5cd1b6e40e5b760c7def8844f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:27.694267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6e550d72-4abe-4eba-9c54-3a31663dab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:27.729821 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ce41e-4edf-485e-9c27-2407646a32f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.730539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ce41e-4edf-485e-9c27-2407646a32f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.730823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ce41e-4edf-485e-9c27-2407646a32f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.731094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ce41e-4edf-485e-9c27-2407646a32f8 admin admin] PATH_INFO: `/v3/projects/e598273e91e949fc8dea71f562dffa83/users/949029f87d404d8bb790c56966cf99d1/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.733488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ce41e-4edf-485e-9c27-2407646a32f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e598273e91e949fc8dea71f562dffa83, user_id=949029f87d404d8bb790c56966cf99d1, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.736140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ce41e-4edf-485e-9c27-2407646a32f8 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.748755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1252/2547] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:27 2022] PUT /identity/v3/projects/e598273e91e949fc8dea71f562dffa83/users/949029f87d404d8bb790c56966cf99d1/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:27.760421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51155bea-47e1-4f88-b666-6777bd5757ee None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.760731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51155bea-47e1-4f88-b666-6777bd5757ee None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.761024 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51155bea-47e1-4f88-b666-6777bd5757ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.773884 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1296/2548] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:27.786790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e550d72-4abe-4eba-9c54-3a31663dab9c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:27.788187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6e550d72-4abe-4eba-9c54-3a31663dab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:27.794905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-51155bea-47e1-4f88-b666-6777bd5757ee None None] MFA Rules not processed for user `949029f87d404d8bb790c56966cf99d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:27.796104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-51155bea-47e1-4f88-b666-6777bd5757ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:27.836213 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:27.849122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e550d72-4abe-4eba-9c54-3a31663dab9c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:27.875551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1253/2549] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:27.901159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf57860-be28-41e6-8d22-737a3d2a0999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:27.901668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf57860-be28-41e6-8d22-737a3d2a0999 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:27.901941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf57860-be28-41e6-8d22-737a3d2a0999 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:27.902207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf57860-be28-41e6-8d22-737a3d2a0999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:27.965701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf57860-be28-41e6-8d22-737a3d2a0999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:27.967148 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf57860-be28-41e6-8d22-737a3d2a0999 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:27.976697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1297/2550] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:28.123805 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51155bea-47e1-4f88-b666-6777bd5757ee None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:28.133926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51155bea-47e1-4f88-b666-6777bd5757ee None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:28.138100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059852ce-7f56-407a-a8f5-db71701d3d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:28.138726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059852ce-7f56-407a-a8f5-db71701d3d64 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:28.139131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059852ce-7f56-407a-a8f5-db71701d3d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:28.139414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059852ce-7f56-407a-a8f5-db71701d3d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:28.148602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059852ce-7f56-407a-a8f5-db71701d3d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:28.150694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059852ce-7f56-407a-a8f5-db71701d3d64 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:28.160523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1254/2551] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:28.748133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf57860-be28-41e6-8d22-737a3d2a0999 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:28.749588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-caf57860-be28-41e6-8d22-737a3d2a0999 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:28.761359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf57860-be28-41e6-8d22-737a3d2a0999 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:28.765021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fba4c-499b-4fd6-902e-a72a5c85e517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:28.765619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fba4c-499b-4fd6-902e-a72a5c85e517 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:28.765894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fba4c-499b-4fd6-902e-a72a5c85e517 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:28.766167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fba4c-499b-4fd6-902e-a72a5c85e517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:28.947835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fba4c-499b-4fd6-902e-a72a5c85e517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:28.949463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fba4c-499b-4fd6-902e-a72a5c85e517 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:28.958340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1298/2552] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:29.187288 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059852ce-7f56-407a-a8f5-db71701d3d64 service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:29.188825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-059852ce-7f56-407a-a8f5-db71701d3d64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:29.257660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059852ce-7f56-407a-a8f5-db71701d3d64 service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:29.262560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:29.263197 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:29.264853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:29.264853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] PATH_INFO: `/v3/users/41d978415a4146d3b63c9904e8cf6d8c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:29.264853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d978415a4146d3b63c9904e8cf6d8c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:29.266784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:29.272507 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:29.680156 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d978415a4146d3b63c9904e8cf6d8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:29.684602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d978415a4146d3b63c9904e8cf6d8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:29.684602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d978415a4146d3b63c9904e8cf6d8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:29.709475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d978415a4146d3b63c9904e8cf6d8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:29.746616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d978415a4146d3b63c9904e8cf6d8c'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:29.839409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1255/2553] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:29 2022] DELETE /identity/v3/users/41d978415a4146d3b63c9904e8cf6d8c => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:29.844313 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ca11b-4c6a-4885-98ff-89a4f0fd9c61 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:29.844756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ca11b-4c6a-4885-98ff-89a4f0fd9c61 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:29.844978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ca11b-4c6a-4885-98ff-89a4f0fd9c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:29.865311 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6b8ca11b-4c6a-4885-98ff-89a4f0fd9c61 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:29.871006 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6b8ca11b-4c6a-4885-98ff-89a4f0fd9c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:29.914956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:29.929593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1299/2554] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:29.940189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:29.941326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:30.006972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0437ec3-538f-473d-8123-325ceaaf75de admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:30.011449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6da58-3321-4582-b81f-51ccd9e42f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:30.012049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6da58-3321-4582-b81f-51ccd9e42f23 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:30.012326 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6da58-3321-4582-b81f-51ccd9e42f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:30.012593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6da58-3321-4582-b81f-51ccd9e42f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:30.022549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6da58-3321-4582-b81f-51ccd9e42f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:30.023984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6da58-3321-4582-b81f-51ccd9e42f23 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:30.033971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1256/2555] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:30.137399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ca11b-4c6a-4885-98ff-89a4f0fd9c61 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:30.140228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6b8ca11b-4c6a-4885-98ff-89a4f0fd9c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:30.151225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ca11b-4c6a-4885-98ff-89a4f0fd9c61 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:30.155808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:30.156425 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:30.156693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:30.156996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] PATH_INFO: `/v3/users/c587b84384304a5689d1f3f8966bd9ce` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:30.158811 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c587b84384304a5689d1f3f8966bd9ce)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:30.161246 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:30.370424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c587b84384304a5689d1f3f8966bd9ce'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.457976 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c587b84384304a5689d1f3f8966bd9ce'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.470836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c587b84384304a5689d1f3f8966bd9ce'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.471499 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c587b84384304a5689d1f3f8966bd9ce'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.478811 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c587b84384304a5689d1f3f8966bd9ce'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.485147 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1300/2556] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:30 2022] DELETE /identity/v3/users/c587b84384304a5689d1f3f8966bd9ce => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:30.489790 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041519-6d78-4b1e-85a1-b8e51df377c1 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:30.490325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041519-6d78-4b1e-85a1-b8e51df377c1 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:30.490325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041519-6d78-4b1e-85a1-b8e51df377c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:30.502288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:30.503850 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:30.514026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6d041519-6d78-4b1e-85a1-b8e51df377c1 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:30.525890 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6d041519-6d78-4b1e-85a1-b8e51df377c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:30.574374 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae415708-6dfc-420f-b920-474e80635e04 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:30.579896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:30.580631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:30.581009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:30.581579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] PATH_INFO: `/v3/projects/584a03e1f44840368199fdcb3f4c29da` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:30.582692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=584a03e1f44840368199fdcb3f4c29da)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:30.584698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:30.590539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1257/2557] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:30.601867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d041519-6d78-4b1e-85a1-b8e51df377c1 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:30.603770 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6d041519-6d78-4b1e-85a1-b8e51df377c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:30.635402 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:30.661812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d041519-6d78-4b1e-85a1-b8e51df377c1 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:30.665616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132d95b-4bbb-4745-a41e-2e46e0a76dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:30.667673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132d95b-4bbb-4745-a41e-2e46e0a76dbb service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:30.667673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132d95b-4bbb-4745-a41e-2e46e0a76dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:30.667673 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132d95b-4bbb-4745-a41e-2e46e0a76dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:30.676657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132d95b-4bbb-4745-a41e-2e46e0a76dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:30.678123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132d95b-4bbb-4745-a41e-2e46e0a76dbb service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:30.687662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1258/2558] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:30.706376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '584a03e1f44840368199fdcb3f4c29da', 'request_id': 'req-b138a9b0-6184-4e44-b125-f3e1782d5db8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.714150 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '584a03e1f44840368199fdcb3f4c29da', 'request_id': 'req-b138a9b0-6184-4e44-b125-f3e1782d5db8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.822118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '584a03e1f44840368199fdcb3f4c29da', 'request_id': 'req-b138a9b0-6184-4e44-b125-f3e1782d5db8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.824596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:30.833808 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] The token cache is being invalidate because project 584a03e1f44840368199fdcb3f4c29da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:30.834107 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.834763 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:30.836733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1301/2559] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:30 2022] DELETE /identity/v3/projects/584a03e1f44840368199fdcb3f4c29da => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:30.841378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132d95b-4bbb-4745-a41e-2e46e0a76dbb service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:30.851664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132d95b-4bbb-4745-a41e-2e46e0a76dbb service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:30.918885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:30.919691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:30.920054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:30.920325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] PATH_INFO: `/v3/users/74918d7253f5468ea42b47bfbe2ad3bc` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:30.921529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] RBAC: Authorizing `identity:delete_user(user_id=74918d7253f5468ea42b47bfbe2ad3bc)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:30.924471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:31.117623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74918d7253f5468ea42b47bfbe2ad3bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.118285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74918d7253f5468ea42b47bfbe2ad3bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.119403 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74918d7253f5468ea42b47bfbe2ad3bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.126553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74918d7253f5468ea42b47bfbe2ad3bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.142127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:31.143710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:31.146612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74918d7253f5468ea42b47bfbe2ad3bc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.161179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138a9b0-6184-4e44-b125-f3e1782d5db8 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:31.165989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:31.166654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:31.166927 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:31.167190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] PATH_INFO: `/v3/projects/285bd7824c8542b7b57fa485e6923b29` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:31.168322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=285bd7824c8542b7b57fa485e6923b29)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:31.170355 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:31.263407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1259/2560] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:30 2022] DELETE /identity/v3/users/74918d7253f5468ea42b47bfbe2ad3bc => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:31.305591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285bd7824c8542b7b57fa485e6923b29', 'request_id': 'req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.312257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285bd7824c8542b7b57fa485e6923b29', 'request_id': 'req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.418893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285bd7824c8542b7b57fa485e6923b29', 'request_id': 'req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.430336 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] The token cache is being invalidate because project 285bd7824c8542b7b57fa485e6923b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:31.430667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.431908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.436569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1302/2561] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:31 2022] DELETE /identity/v3/projects/285bd7824c8542b7b57fa485e6923b29 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:31.437736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:31.439135 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:31.492170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:31.506337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fde02-ba71-4678-9b64-7257264d399c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:31.581177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:31.582067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:31.582429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:31.582757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] PATH_INFO: `/v3/users/949029f87d404d8bb790c56966cf99d1` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:31.583904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=949029f87d404d8bb790c56966cf99d1)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:31.585869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:31.794063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:31.796405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:31.804112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '949029f87d404d8bb790c56966cf99d1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.804781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '949029f87d404d8bb790c56966cf99d1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.805550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '949029f87d404d8bb790c56966cf99d1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.813747 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60579ea-5020-44a3-ae02-f40a4ba1eb0a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:31.815806 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '949029f87d404d8bb790c56966cf99d1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.819043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:31.819787 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:31.820172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:31.820566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] PATH_INFO: `/v3/projects/93b9bd02ab13451b8135b5fd0603b4d0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:31.822065 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b9bd02ab13451b8135b5fd0603b4d0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:31.824069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:31.829679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '949029f87d404d8bb790c56966cf99d1'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.950645 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1260/2562] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:31 2022] DELETE /identity/v3/users/949029f87d404d8bb790c56966cf99d1 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:31.971962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b9bd02ab13451b8135b5fd0603b4d0', 'request_id': 'req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:31.980021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b9bd02ab13451b8135b5fd0603b4d0', 'request_id': 'req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.051334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b9bd02ab13451b8135b5fd0603b4d0', 'request_id': 'req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.066434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] The token cache is being invalidate because project 93b9bd02ab13451b8135b5fd0603b4d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:32.066819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.067554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.069104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1303/2563] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:31 2022] DELETE /identity/v3/projects/93b9bd02ab13451b8135b5fd0603b4d0 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:32.074070 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:32.075971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:32.143276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f805cc-76ee-409a-8466-0508a8410b69 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:32.148004 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:32.148617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:32.148931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:32.149205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] PATH_INFO: `/v3/users/515b99f46cd34acca631bc75c091a706` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:32.151041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=515b99f46cd34acca631bc75c091a706)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:32.153042 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:32.175576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:32.475472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '515b99f46cd34acca631bc75c091a706'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.476057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '515b99f46cd34acca631bc75c091a706'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.476751 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '515b99f46cd34acca631bc75c091a706'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.484240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '515b99f46cd34acca631bc75c091a706'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.495858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-5d491e20-ac3d-4247-a2c0-44d3fe43bd80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '515b99f46cd34acca631bc75c091a706'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.527253 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:32.529010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:32.597581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:32.601520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b957a-11c9-4a6b-81cc-b4c9e3a1c367 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:32.606319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:32.606973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:32.607251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:32.607523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] PATH_INFO: `/v3/projects/e598273e91e949fc8dea71f562dffa83` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:32.608636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e598273e91e949fc8dea71f562dffa83)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:32.610613 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:32.634868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1261/2564] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:32 2022] DELETE /identity/v3/users/515b99f46cd34acca631bc75c091a706 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:32.764173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e598273e91e949fc8dea71f562dffa83', 'request_id': 'req-29d8d279-fd71-4f90-a411-ef701aec9368'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.770847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e598273e91e949fc8dea71f562dffa83', 'request_id': 'req-29d8d279-fd71-4f90-a411-ef701aec9368'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.904423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e598273e91e949fc8dea71f562dffa83', 'request_id': 'req-29d8d279-fd71-4f90-a411-ef701aec9368'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.919066 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] The token cache is being invalidate because project e598273e91e949fc8dea71f562dffa83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:32.919680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.920511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-29d8d279-fd71-4f90-a411-ef701aec9368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:32.925125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1304/2565] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:32 2022] DELETE /identity/v3/projects/e598273e91e949fc8dea71f562dffa83 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:32.930556 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d593d7-c778-4756-bfd7-1f7102f84882 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:32.931744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d593d7-c778-4756-bfd7-1f7102f84882 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:32.932046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d593d7-c778-4756-bfd7-1f7102f84882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:32.957140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-78d593d7-c778-4756-bfd7-1f7102f84882 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:32.958360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-78d593d7-c778-4756-bfd7-1f7102f84882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:33.024358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1262/2566] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:33.032505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05699d90-5cfa-4615-84e3-19ff888c4496 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.037467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05699d90-5cfa-4615-84e3-19ff888c4496 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.037467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05699d90-5cfa-4615-84e3-19ff888c4496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.053870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-05699d90-5cfa-4615-84e3-19ff888c4496 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:33.055082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-05699d90-5cfa-4615-84e3-19ff888c4496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:33.063950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1305/2567] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:33.068187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d593d7-c778-4756-bfd7-1f7102f84882 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.078698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d593d7-c778-4756-bfd7-1f7102f84882 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.083217 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.083827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.084138 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.084400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.085080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:33.085378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.086961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.102676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1263/2568] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:46:33.103990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05699d90-5cfa-4615-84e3-19ff888c4496 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.113434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05699d90-5cfa-4615-84e3-19ff888c4496 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.119271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd6bda-73ba-4e9f-b838-725abadcbe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.120027 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd6bda-73ba-4e9f-b838-725abadcbe3c admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.120303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd6bda-73ba-4e9f-b838-725abadcbe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.120570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd6bda-73ba-4e9f-b838-725abadcbe3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.121237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd6bda-73ba-4e9f-b838-725abadcbe3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.123184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd6bda-73ba-4e9f-b838-725abadcbe3c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.160954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.163282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:33.175841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b4d0b-e33e-4ffa-adb0-809293e948aa admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.180741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.182166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.182423 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.182697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] PATH_INFO: `/v3/projects/53a06391f9874dc6b042bbf5b7e4d6c1` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.183977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a06391f9874dc6b042bbf5b7e4d6c1)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.186032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.240647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1306/2569] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:33 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:33.245813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd6bda-73ba-4e9f-b838-725abadcbe3c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.305972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd6bda-73ba-4e9f-b838-725abadcbe3c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.315966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bf0aa-3b4a-4287-a3b1-a15334cd9ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.315966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bf0aa-3b4a-4287-a3b1-a15334cd9ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.315966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bf0aa-3b4a-4287-a3b1-a15334cd9ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.315966 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001bf0aa-3b4a-4287-a3b1-a15334cd9ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.316358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bf0aa-3b4a-4287-a3b1-a15334cd9ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.316358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001bf0aa-3b4a-4287-a3b1-a15334cd9ed1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.361855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a06391f9874dc6b042bbf5b7e4d6c1', 'request_id': 'req-4cdb8b57-673f-4e23-b410-68bb6c21c28d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:33.367325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a06391f9874dc6b042bbf5b7e4d6c1', 'request_id': 'req-4cdb8b57-673f-4e23-b410-68bb6c21c28d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:33.468283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1307/2570] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:33 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:33.472076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bf0aa-3b4a-4287-a3b1-a15334cd9ed1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.482566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001bf0aa-3b4a-4287-a3b1-a15334cd9ed1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.503686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a06391f9874dc6b042bbf5b7e4d6c1', 'request_id': 'req-4cdb8b57-673f-4e23-b410-68bb6c21c28d'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:33.517714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] The token cache is being invalidate because project 53a06391f9874dc6b042bbf5b7e4d6c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:33.517995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:33.518779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4cdb8b57-673f-4e23-b410-68bb6c21c28d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:33.520304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1264/2571] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:33 2022] DELETE /identity/v3/projects/53a06391f9874dc6b042bbf5b7e4d6c1 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:33.526346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6298e1a-229b-40e9-b3a3-33f689ba52a0 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.527079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6298e1a-229b-40e9-b3a3-33f689ba52a0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.527413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6298e1a-229b-40e9-b3a3-33f689ba52a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.530251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:33.541221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:33.541822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62dd2d-e9e7-45a9-b485-540e3ba1a925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.542676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62dd2d-e9e7-45a9-b485-540e3ba1a925 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.542848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62dd2d-e9e7-45a9-b485-540e3ba1a925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.543119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62dd2d-e9e7-45a9-b485-540e3ba1a925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.543812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62dd2d-e9e7-45a9-b485-540e3ba1a925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.545382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62dd2d-e9e7-45a9-b485-540e3ba1a925 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.548251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e6298e1a-229b-40e9-b3a3-33f689ba52a0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:33.549581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e6298e1a-229b-40e9-b3a3-33f689ba52a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:33.555314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1308/2572] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:33.559745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1265/2573] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:33.560712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62dd2d-e9e7-45a9-b485-540e3ba1a925 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.563062 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5a62dd2d-e9e7-45a9-b485-540e3ba1a925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:33.568015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6888c-d02e-484a-ac31-4bb29be4560f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.569903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6888c-d02e-484a-ac31-4bb29be4560f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.569903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6888c-d02e-484a-ac31-4bb29be4560f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.577301 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62dd2d-e9e7-45a9-b485-540e3ba1a925 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.581942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a91b3-f9ea-4da4-853d-51cfbca7306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.582562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a91b3-f9ea-4da4-853d-51cfbca7306a admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.582841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a91b3-f9ea-4da4-853d-51cfbca7306a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.583118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a91b3-f9ea-4da4-853d-51cfbca7306a admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.583755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a91b3-f9ea-4da4-853d-51cfbca7306a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.585385 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a91b3-f9ea-4da4-853d-51cfbca7306a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.588436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-35f6888c-d02e-484a-ac31-4bb29be4560f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:33.590360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-35f6888c-d02e-484a-ac31-4bb29be4560f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:33.597733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1309/2574] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:33.598879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1266/2575] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:33.601539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a91b3-f9ea-4da4-853d-51cfbca7306a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.602961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6888c-d02e-484a-ac31-4bb29be4560f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.612476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a91b3-f9ea-4da4-853d-51cfbca7306a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.613179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6888c-d02e-484a-ac31-4bb29be4560f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.617139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2443b-af56-44a0-9ec4-2591dd85a5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.617817 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2443b-af56-44a0-9ec4-2591dd85a5de admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.618005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112fa8-1bd2-4ee5-b038-f6706e361f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.618139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2443b-af56-44a0-9ec4-2591dd85a5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.618348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2443b-af56-44a0-9ec4-2591dd85a5de admin admin] PATH_INFO: `/v3/projects/8814597975704d4eb3dba73e7d31859c/users/fd17d21e7e674bc3a01c556b4ddbf8e5/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.618596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112fa8-1bd2-4ee5-b038-f6706e361f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.618867 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112fa8-1bd2-4ee5-b038-f6706e361f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.619137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9112fa8-1bd2-4ee5-b038-f6706e361f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.619769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112fa8-1bd2-4ee5-b038-f6706e361f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:33.621506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112fa8-1bd2-4ee5-b038-f6706e361f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.622594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9112fa8-1bd2-4ee5-b038-f6706e361f41 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.634644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1267/2576] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:46:33.634810 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2443b-af56-44a0-9ec4-2591dd85a5de admin admin] RBAC: Authorizing `identity:create_grant(project_id=8814597975704d4eb3dba73e7d31859c, user_id=fd17d21e7e674bc3a01c556b4ddbf8e5, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.637998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2443b-af56-44a0-9ec4-2591dd85a5de admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.639690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112fa8-1bd2-4ee5-b038-f6706e361f41 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.649851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9112fa8-1bd2-4ee5-b038-f6706e361f41 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.654660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36f335-ecdb-42b2-85b5-41bbdeb26f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.655274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36f335-ecdb-42b2-85b5-41bbdeb26f20 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.655548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36f335-ecdb-42b2-85b5-41bbdeb26f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.655817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36f335-ecdb-42b2-85b5-41bbdeb26f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.656439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36f335-ecdb-42b2-85b5-41bbdeb26f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.658376 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36f335-ecdb-42b2-85b5-41bbdeb26f20 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.720544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1310/2577] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:33 2022] PUT /identity/v3/projects/8814597975704d4eb3dba73e7d31859c/users/fd17d21e7e674bc3a01c556b4ddbf8e5/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:33.731940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346185a2-bc5d-491d-9b33-f1f4dd0c32d5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.732228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346185a2-bc5d-491d-9b33-f1f4dd0c32d5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.732514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346185a2-bc5d-491d-9b33-f1f4dd0c32d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.766342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-346185a2-bc5d-491d-9b33-f1f4dd0c32d5 None None] MFA Rules not processed for user `fd17d21e7e674bc3a01c556b4ddbf8e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:33.766342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-346185a2-bc5d-491d-9b33-f1f4dd0c32d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:33.799366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1268/2578] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:33 2022] POST /identity/v3/projects => generated 365 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:33.803452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36f335-ecdb-42b2-85b5-41bbdeb26f20 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.838741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1311/2579] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:33.842940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346185a2-bc5d-491d-9b33-f1f4dd0c32d5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:33.895760 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36f335-ecdb-42b2-85b5-41bbdeb26f20 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.901355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748fc9b-d858-41d2-b9de-2f957e14dde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.902258 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748fc9b-d858-41d2-b9de-2f957e14dde3 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.902670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748fc9b-d858-41d2-b9de-2f957e14dde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.902948 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748fc9b-d858-41d2-b9de-2f957e14dde3 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.903574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748fc9b-d858-41d2-b9de-2f957e14dde3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.905536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748fc9b-d858-41d2-b9de-2f957e14dde3 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:33.918608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346185a2-bc5d-491d-9b33-f1f4dd0c32d5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:33.923195 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5fb15-e15f-4142-970c-3ea4101bd7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:33.923789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5fb15-e15f-4142-970c-3ea4101bd7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:33.924060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5fb15-e15f-4142-970c-3ea4101bd7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:33.924566 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5fb15-e15f-4142-970c-3ea4101bd7f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:33.925216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5fb15-e15f-4142-970c-3ea4101bd7f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:33.927476 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5fb15-e15f-4142-970c-3ea4101bd7f1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.089649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1269/2580] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:33 2022] POST /identity/v3/users => generated 283 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:34.089891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1312/2581] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:33 2022] POST /identity/v3/projects => generated 365 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:34.091176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748fc9b-d858-41d2-b9de-2f957e14dde3 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.091676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5fb15-e15f-4142-970c-3ea4101bd7f1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.102335 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748fc9b-d858-41d2-b9de-2f957e14dde3 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.161469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454cba5-3aff-42e9-a56e-5d7a0bcfa952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.161774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5fb15-e15f-4142-970c-3ea4101bd7f1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.162081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454cba5-3aff-42e9-a56e-5d7a0bcfa952 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.162364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454cba5-3aff-42e9-a56e-5d7a0bcfa952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.162629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454cba5-3aff-42e9-a56e-5d7a0bcfa952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.163269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454cba5-3aff-42e9-a56e-5d7a0bcfa952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.164889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454cba5-3aff-42e9-a56e-5d7a0bcfa952 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.167304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb544c-c75c-460a-9169-fa39c7acee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.168087 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb544c-c75c-460a-9169-fa39c7acee80 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.168725 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb544c-c75c-460a-9169-fa39c7acee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.169133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb544c-c75c-460a-9169-fa39c7acee80 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.169694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb544c-c75c-460a-9169-fa39c7acee80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.171609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb544c-c75c-460a-9169-fa39c7acee80 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.175309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1270/2582] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:34.179632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454cba5-3aff-42e9-a56e-5d7a0bcfa952 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.190609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454cba5-3aff-42e9-a56e-5d7a0bcfa952 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.195274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479ce63-c877-434a-90d0-4171b6a5e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.195878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479ce63-c877-434a-90d0-4171b6a5e611 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.196157 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479ce63-c877-434a-90d0-4171b6a5e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.196430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479ce63-c877-434a-90d0-4171b6a5e611 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.197091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479ce63-c877-434a-90d0-4171b6a5e611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.198997 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479ce63-c877-434a-90d0-4171b6a5e611 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.209679 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1271/2583] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:34.217243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479ce63-c877-434a-90d0-4171b6a5e611 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.226677 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479ce63-c877-434a-90d0-4171b6a5e611 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.231844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d341efe-0482-439e-aa18-7250f3d2722a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.232471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d341efe-0482-439e-aa18-7250f3d2722a admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.232746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d341efe-0482-439e-aa18-7250f3d2722a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.233044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d341efe-0482-439e-aa18-7250f3d2722a admin admin] PATH_INFO: `/v3/projects/45a0e67cefb64a8983aa4220e75cf370/users/c814f5342283472aab1a4d0b71f3f6c3/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.248322 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d341efe-0482-439e-aa18-7250f3d2722a admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a0e67cefb64a8983aa4220e75cf370, user_id=c814f5342283472aab1a4d0b71f3f6c3, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.251319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d341efe-0482-439e-aa18-7250f3d2722a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.290644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1313/2584] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:34 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:34.294952 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb544c-c75c-460a-9169-fa39c7acee80 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.309947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb544c-c75c-460a-9169-fa39c7acee80 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.313935 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcab88-24c5-4999-969a-0b35cdcac83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.314632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcab88-24c5-4999-969a-0b35cdcac83b admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.314923 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcab88-24c5-4999-969a-0b35cdcac83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.315202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bcab88-24c5-4999-969a-0b35cdcac83b admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.315862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcab88-24c5-4999-969a-0b35cdcac83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.317513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bcab88-24c5-4999-969a-0b35cdcac83b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.330178 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1314/2585] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:34.334340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcab88-24c5-4999-969a-0b35cdcac83b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.346165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bcab88-24c5-4999-969a-0b35cdcac83b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.351702 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0577fa4-1d11-4417-9278-49cbaff43c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.351953 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0577fa4-1d11-4417-9278-49cbaff43c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.352209 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0577fa4-1d11-4417-9278-49cbaff43c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.352480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0577fa4-1d11-4417-9278-49cbaff43c21 admin admin] PATH_INFO: `/v3/projects/92e684d84b394ae6979228c0f7f97fcf/users/94e3a1aabbc24f11b0d95319463a2e9b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.368826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0577fa4-1d11-4417-9278-49cbaff43c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e684d84b394ae6979228c0f7f97fcf, user_id=94e3a1aabbc24f11b0d95319463a2e9b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.371533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0577fa4-1d11-4417-9278-49cbaff43c21 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.390238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1272/2586] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:34 2022] PUT /identity/v3/projects/45a0e67cefb64a8983aa4220e75cf370/users/c814f5342283472aab1a4d0b71f3f6c3/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:34.402180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512bae0-d950-439b-a4b1-02c8f1d602d0 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.402533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512bae0-d950-439b-a4b1-02c8f1d602d0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.402872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512bae0-d950-439b-a4b1-02c8f1d602d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.431451 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-5512bae0-d950-439b-a4b1-02c8f1d602d0 None None] MFA Rules not processed for user `c814f5342283472aab1a4d0b71f3f6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:34.432703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5512bae0-d950-439b-a4b1-02c8f1d602d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:34.497564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1273/2587] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:34.502348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512bae0-d950-439b-a4b1-02c8f1d602d0 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.531960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1315/2588] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:34 2022] PUT /identity/v3/projects/92e684d84b394ae6979228c0f7f97fcf/users/94e3a1aabbc24f11b0d95319463a2e9b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:34.535841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0577fa4-1d11-4417-9278-49cbaff43c21 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.569683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512bae0-d950-439b-a4b1-02c8f1d602d0 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.599830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0577fa4-1d11-4417-9278-49cbaff43c21 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.605118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116a3a5-3374-4c4b-b307-d41092de9a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.606048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116a3a5-3374-4c4b-b307-d41092de9a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.606329 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116a3a5-3374-4c4b-b307-d41092de9a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.606595 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116a3a5-3374-4c4b-b307-d41092de9a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.607231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116a3a5-3374-4c4b-b307-d41092de9a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.608831 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116a3a5-3374-4c4b-b307-d41092de9a01 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.618889 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1316/2589] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:34.623012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116a3a5-3374-4c4b-b307-d41092de9a01 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.629347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9a312-481d-451d-ad8d-d2bcbcac68eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.629962 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9a312-481d-451d-ad8d-d2bcbcac68eb admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.630234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9a312-481d-451d-ad8d-d2bcbcac68eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.630501 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9a312-481d-451d-ad8d-d2bcbcac68eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.631114 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9a312-481d-451d-ad8d-d2bcbcac68eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.633108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9a312-481d-451d-ad8d-d2bcbcac68eb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.633236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116a3a5-3374-4c4b-b307-d41092de9a01 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.637904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0b691-a9aa-43ab-bf01-01bb2ee76bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.638847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b691-a9aa-43ab-bf01-01bb2ee76bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.638847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b691-a9aa-43ab-bf01-01bb2ee76bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.639072 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b691-a9aa-43ab-bf01-01bb2ee76bc1 admin admin] PATH_INFO: `/v3/projects/92e684d84b394ae6979228c0f7f97fcf/users/94e3a1aabbc24f11b0d95319463a2e9b/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.641019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b691-a9aa-43ab-bf01-01bb2ee76bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e684d84b394ae6979228c0f7f97fcf, user_id=94e3a1aabbc24f11b0d95319463a2e9b, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.643945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0b691-a9aa-43ab-bf01-01bb2ee76bc1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.659672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1317/2590] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:34 2022] PUT /identity/v3/projects/92e684d84b394ae6979228c0f7f97fcf/users/94e3a1aabbc24f11b0d95319463a2e9b/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:34.671020 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6217e-c97d-408e-96ce-ff7ed244698e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.671302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6217e-c97d-408e-96ce-ff7ed244698e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.671612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6217e-c97d-408e-96ce-ff7ed244698e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.701821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2ef6217e-c97d-408e-96ce-ff7ed244698e None None] MFA Rules not processed for user `94e3a1aabbc24f11b0d95319463a2e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:34.703287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2ef6217e-c97d-408e-96ce-ff7ed244698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:34.788518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1318/2591] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:34.801480 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6217e-c97d-408e-96ce-ff7ed244698e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:34.802881 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2ef6217e-c97d-408e-96ce-ff7ed244698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:34.847598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:34.862781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6217e-c97d-408e-96ce-ff7ed244698e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:34.866424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a909b2a-1271-40a4-b016-423602eeca0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:34.867026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a909b2a-1271-40a4-b016-423602eeca0e service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:34.867303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a909b2a-1271-40a4-b016-423602eeca0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:34.867571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a909b2a-1271-40a4-b016-423602eeca0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:34.876996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a909b2a-1271-40a4-b016-423602eeca0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:34.878426 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a909b2a-1271-40a4-b016-423602eeca0e service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:34.888420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1319/2592] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:34.977415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1274/2593] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:34 2022] POST /identity/v3/projects => generated 367 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:34.981544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a909b2a-1271-40a4-b016-423602eeca0e service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:35.047596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a909b2a-1271-40a4-b016-423602eeca0e service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:35.103675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826dad2-573a-4783-932a-70325c061545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:35.104277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826dad2-573a-4783-932a-70325c061545 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:35.104549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826dad2-573a-4783-932a-70325c061545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:35.104835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826dad2-573a-4783-932a-70325c061545 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:35.105496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826dad2-573a-4783-932a-70325c061545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:35.107387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826dad2-573a-4783-932a-70325c061545 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:35.197340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9a312-481d-451d-ad8d-d2bcbcac68eb admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:35.205406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:35.213527 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1320/2594] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:34 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:35.217114 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826dad2-573a-4783-932a-70325c061545 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:35.229870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826dad2-573a-4783-932a-70325c061545 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:35.236941 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f163382-bea0-4c08-98de-f3b3550ac3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:35.237703 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f163382-bea0-4c08-98de-f3b3550ac3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:35.238076 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f163382-bea0-4c08-98de-f3b3550ac3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:35.238455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f163382-bea0-4c08-98de-f3b3550ac3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:35.239242 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f163382-bea0-4c08-98de-f3b3550ac3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:35.240967 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f163382-bea0-4c08-98de-f3b3550ac3b1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:35.254203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1321/2595] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:35.261852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f163382-bea0-4c08-98de-f3b3550ac3b1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:35.265969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9a312-481d-451d-ad8d-d2bcbcac68eb admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:35.269989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c100-ecf9-4c84-ae61-230efdec9160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:35.270628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c100-ecf9-4c84-ae61-230efdec9160 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:35.270905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c100-ecf9-4c84-ae61-230efdec9160 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:35.271295 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c100-ecf9-4c84-ae61-230efdec9160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:35.275409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f163382-bea0-4c08-98de-f3b3550ac3b1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:35.281417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59fbd8-8742-4959-989f-952f5ef7df5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:35.282120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59fbd8-8742-4959-989f-952f5ef7df5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:35.282377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59fbd8-8742-4959-989f-952f5ef7df5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:35.282661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59fbd8-8742-4959-989f-952f5ef7df5d admin admin] PATH_INFO: `/v3/projects/25261c0c5fc14e2d8d4095efe3f93c18/users/ee2d7ee7dfcf4183b47dd2760201ba31/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:35.305917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59fbd8-8742-4959-989f-952f5ef7df5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=25261c0c5fc14e2d8d4095efe3f93c18, user_id=ee2d7ee7dfcf4183b47dd2760201ba31, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:35.309067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59fbd8-8742-4959-989f-952f5ef7df5d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:35.351118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62c100-ecf9-4c84-ae61-230efdec9160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:35.352571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62c100-ecf9-4c84-ae61-230efdec9160 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:35.361916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1275/2596] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:35.424104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1322/2597] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:35 2022] PUT /identity/v3/projects/25261c0c5fc14e2d8d4095efe3f93c18/users/ee2d7ee7dfcf4183b47dd2760201ba31/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:35.428309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c100-ecf9-4c84-ae61-230efdec9160 service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:35.439739 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c100-ecf9-4c84-ae61-230efdec9160 service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:35.498312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b79d69-8b27-4bfc-ae7b-911670381662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:35.498914 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b79d69-8b27-4bfc-ae7b-911670381662 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:35.499191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b79d69-8b27-4bfc-ae7b-911670381662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:35.499463 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b79d69-8b27-4bfc-ae7b-911670381662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:35.500103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b79d69-8b27-4bfc-ae7b-911670381662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:35.501983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b79d69-8b27-4bfc-ae7b-911670381662 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:35.513273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1276/2598] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:35.517064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59fbd8-8742-4959-989f-952f5ef7df5d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:35.527917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59fbd8-8742-4959-989f-952f5ef7df5d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:35.533272 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb6608-25f6-43be-bf6a-125bad71e2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:35.534004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb6608-25f6-43be-bf6a-125bad71e2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:35.534310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb6608-25f6-43be-bf6a-125bad71e2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:35.534583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb6608-25f6-43be-bf6a-125bad71e2ac admin admin] PATH_INFO: `/v3/projects/25261c0c5fc14e2d8d4095efe3f93c18/users/ee2d7ee7dfcf4183b47dd2760201ba31/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:35.536523 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb6608-25f6-43be-bf6a-125bad71e2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=25261c0c5fc14e2d8d4095efe3f93c18, user_id=ee2d7ee7dfcf4183b47dd2760201ba31, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:35.539269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb6608-25f6-43be-bf6a-125bad71e2ac admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:35.550731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1323/2599] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:35 2022] PUT /identity/v3/projects/25261c0c5fc14e2d8d4095efe3f93c18/users/ee2d7ee7dfcf4183b47dd2760201ba31/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:35.563053 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e575891-2a49-46ec-b92a-42d7641bb679 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:35.563340 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e575891-2a49-46ec-b92a-42d7641bb679 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:35.563641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e575891-2a49-46ec-b92a-42d7641bb679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:35.593924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-0e575891-2a49-46ec-b92a-42d7641bb679 None None] MFA Rules not processed for user `ee2d7ee7dfcf4183b47dd2760201ba31`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:35.595190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0e575891-2a49-46ec-b92a-42d7641bb679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:35.670792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1277/2600] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:35.673581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:35.682570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb6608-25f6-43be-bf6a-125bad71e2ac admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:35.685118 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ecdb6608-25f6-43be-bf6a-125bad71e2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:35.745317 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb6608-25f6-43be-bf6a-125bad71e2ac admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:35.750095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52630266-88b5-49b5-9e8c-d9338eb5c77b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:35.750715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52630266-88b5-49b5-9e8c-d9338eb5c77b service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:35.751247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52630266-88b5-49b5-9e8c-d9338eb5c77b service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:35.751515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52630266-88b5-49b5-9e8c-d9338eb5c77b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:35.768985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52630266-88b5-49b5-9e8c-d9338eb5c77b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:35.771229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52630266-88b5-49b5-9e8c-d9338eb5c77b service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:35.782554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1324/2601] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:35.793816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:36.040902 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e575891-2a49-46ec-b92a-42d7641bb679 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:36.051869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e575891-2a49-46ec-b92a-42d7641bb679 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:36.055509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4636a-9f6e-47e7-a941-96906b1ce0fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:36.056931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4636a-9f6e-47e7-a941-96906b1ce0fb service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:36.056931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4636a-9f6e-47e7-a941-96906b1ce0fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:36.056931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4636a-9f6e-47e7-a941-96906b1ce0fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:36.116210 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4636a-9f6e-47e7-a941-96906b1ce0fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:36.117726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4636a-9f6e-47e7-a941-96906b1ce0fb service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:36.127835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1278/2602] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:36.184540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:36.202059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=100 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:40.114029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52630266-88b5-49b5-9e8c-d9338eb5c77b service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:40.178930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52630266-88b5-49b5-9e8c-d9338eb5c77b service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:40.183880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:40.184493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:40.184764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:40.185057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] PATH_INFO: `/v3/users/fd17d21e7e674bc3a01c556b4ddbf8e5` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:40.186166 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd17d21e7e674bc3a01c556b4ddbf8e5)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:40.188051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:40.444073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd17d21e7e674bc3a01c556b4ddbf8e5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:40.515237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd17d21e7e674bc3a01c556b4ddbf8e5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:40.529328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd17d21e7e674bc3a01c556b4ddbf8e5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:40.530051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd17d21e7e674bc3a01c556b4ddbf8e5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:40.537894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd17d21e7e674bc3a01c556b4ddbf8e5'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:40.540233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:40.546064 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1325/2603] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:40 2022] DELETE /identity/v3/users/fd17d21e7e674bc3a01c556b4ddbf8e5 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:40.550918 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6a810-3f20-4f0d-a066-aab6eaaed425 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:40.551405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6a810-3f20-4f0d-a066-aab6eaaed425 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:40.551519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6a810-3f20-4f0d-a066-aab6eaaed425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:40.579704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9ea6a810-3f20-4f0d-a066-aab6eaaed425 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:40.580975 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9ea6a810-3f20-4f0d-a066-aab6eaaed425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:40.642704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1279/2604] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:40.653446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:40.656958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:40.728744 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a871ec8-5238-4d4c-8765-0da54fe17e7a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:40.737928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47976-ddb7-438d-bedf-0da8fee131dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:40.738550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47976-ddb7-438d-bedf-0da8fee131dd service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:40.738833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47976-ddb7-438d-bedf-0da8fee131dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:40.739102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf47976-ddb7-438d-bedf-0da8fee131dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:40.750579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf47976-ddb7-438d-bedf-0da8fee131dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:40.752445 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf47976-ddb7-438d-bedf-0da8fee131dd service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:40.762792 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1326/2605] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:41.220481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6a810-3f20-4f0d-a066-aab6eaaed425 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:41.221942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9ea6a810-3f20-4f0d-a066-aab6eaaed425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:41.235385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6a810-3f20-4f0d-a066-aab6eaaed425 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:41.240011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:41.240706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:41.240954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:41.241227 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] PATH_INFO: `/v3/projects/8814597975704d4eb3dba73e7d31859c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:41.242511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8814597975704d4eb3dba73e7d31859c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:41.244425 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:41.348444 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8814597975704d4eb3dba73e7d31859c', 'request_id': 'req-ba2bd734-e696-40a5-818f-39e6da0dc3ce'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.354272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8814597975704d4eb3dba73e7d31859c', 'request_id': 'req-ba2bd734-e696-40a5-818f-39e6da0dc3ce'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.501733 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8814597975704d4eb3dba73e7d31859c', 'request_id': 'req-ba2bd734-e696-40a5-818f-39e6da0dc3ce'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.514189 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] The token cache is being invalidate because project 8814597975704d4eb3dba73e7d31859c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:41.514478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.515105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.516606 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1280/2606] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:41 2022] DELETE /identity/v3/projects/8814597975704d4eb3dba73e7d31859c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:41.520777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47976-ddb7-438d-bedf-0da8fee131dd service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:41.533659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf47976-ddb7-438d-bedf-0da8fee131dd service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:41.588932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:41.590214 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:41.590487 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:41.590753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] PATH_INFO: `/v3/users/94e3a1aabbc24f11b0d95319463a2e9b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:41.591903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94e3a1aabbc24f11b0d95319463a2e9b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:41.593820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:41.773451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94e3a1aabbc24f11b0d95319463a2e9b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.852912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94e3a1aabbc24f11b0d95319463a2e9b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.873638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94e3a1aabbc24f11b0d95319463a2e9b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.874549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94e3a1aabbc24f11b0d95319463a2e9b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.882762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f7ed688-640e-444b-935a-680314910774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94e3a1aabbc24f11b0d95319463a2e9b'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:41.890797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1327/2607] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:41 2022] DELETE /identity/v3/users/94e3a1aabbc24f11b0d95319463a2e9b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:42.387298 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:42.388864 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:42.404117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bd734-e696-40a5-818f-39e6da0dc3ce admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:42.448405 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:42.480728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:42.481425 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:42.481820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:42.481944 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] PATH_INFO: `/v3/projects/92e684d84b394ae6979228c0f7f97fcf` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:42.483088 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e684d84b394ae6979228c0f7f97fcf)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:42.485005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:42.646725 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e684d84b394ae6979228c0f7f97fcf', 'request_id': 'req-3b9e12be-42d3-4567-98e4-e00ae87af1ac'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:42.653172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e684d84b394ae6979228c0f7f97fcf', 'request_id': 'req-3b9e12be-42d3-4567-98e4-e00ae87af1ac'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:42.720761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e684d84b394ae6979228c0f7f97fcf', 'request_id': 'req-3b9e12be-42d3-4567-98e4-e00ae87af1ac'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:42.733668 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] The token cache is being invalidate because project 92e684d84b394ae6979228c0f7f97fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:42.733961 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:42.734605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3b9e12be-42d3-4567-98e4-e00ae87af1ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:42.736143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1281/2608] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:42 2022] DELETE /identity/v3/projects/92e684d84b394ae6979228c0f7f97fcf => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:42.741101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c213-6687-4248-9a1e-bd75cbb34818 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:42.741598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c213-6687-4248-9a1e-bd75cbb34818 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:42.741693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9c213-6687-4248-9a1e-bd75cbb34818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:42.763021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-8ab9c213-6687-4248-9a1e-bd75cbb34818 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:42.764248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-8ab9c213-6687-4248-9a1e-bd75cbb34818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:42.782910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:42.826405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1328/2609] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:42.834414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136660e-0a41-4ea8-9138-6247b161c962 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:42.834695 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136660e-0a41-4ea8-9138-6247b161c962 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:42.834970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136660e-0a41-4ea8-9138-6247b161c962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:42.859796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-5136660e-0a41-4ea8-9138-6247b161c962 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:42.859796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-5136660e-0a41-4ea8-9138-6247b161c962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:42.866638 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1282/2610] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:42.871320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9c213-6687-4248-9a1e-bd75cbb34818 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:42.882028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9c213-6687-4248-9a1e-bd75cbb34818 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:42.887749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63d8c6-eaa0-4d65-beb6-9592b24dd9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:42.892216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63d8c6-eaa0-4d65-beb6-9592b24dd9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:42.892216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63d8c6-eaa0-4d65-beb6-9592b24dd9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:42.892216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63d8c6-eaa0-4d65-beb6-9592b24dd9d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:42.892216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63d8c6-eaa0-4d65-beb6-9592b24dd9d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:42.892216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63d8c6-eaa0-4d65-beb6-9592b24dd9d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:42.892216 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63d8c6-eaa0-4d65-beb6-9592b24dd9d6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:42.906274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1329/2611] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:46:42.910827 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136660e-0a41-4ea8-9138-6247b161c962 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:42.921215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136660e-0a41-4ea8-9138-6247b161c962 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:42.925927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1284507-3ef9-4953-b0e2-b3f8b8627ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:42.926523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1284507-3ef9-4953-b0e2-b3f8b8627ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:42.926796 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1284507-3ef9-4953-b0e2-b3f8b8627ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:42.927059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1284507-3ef9-4953-b0e2-b3f8b8627ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:42.927682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1284507-3ef9-4953-b0e2-b3f8b8627ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:42.929685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1284507-3ef9-4953-b0e2-b3f8b8627ef9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:43.026418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1283/2612] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:42 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:43.032327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63d8c6-eaa0-4d65-beb6-9592b24dd9d6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:43.042662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63d8c6-eaa0-4d65-beb6-9592b24dd9d6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:43.100752 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b828d8-3f1f-4f19-86b5-46eff95f58c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:43.101389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b828d8-3f1f-4f19-86b5-46eff95f58c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:43.101662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b828d8-3f1f-4f19-86b5-46eff95f58c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:43.101934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b828d8-3f1f-4f19-86b5-46eff95f58c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:43.103218 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b828d8-3f1f-4f19-86b5-46eff95f58c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:43.105619 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b828d8-3f1f-4f19-86b5-46eff95f58c5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:43.195319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1330/2613] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:43 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:43.200173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1284507-3ef9-4953-b0e2-b3f8b8627ef9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:43.211054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1284507-3ef9-4953-b0e2-b3f8b8627ef9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:43.218127 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0807dd-5b52-48d6-97fb-0b0a1c431486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:43.218743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0807dd-5b52-48d6-97fb-0b0a1c431486 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:43.219015 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0807dd-5b52-48d6-97fb-0b0a1c431486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:43.219281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0807dd-5b52-48d6-97fb-0b0a1c431486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:43.219921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0807dd-5b52-48d6-97fb-0b0a1c431486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:43.221583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0807dd-5b52-48d6-97fb-0b0a1c431486 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:43.232163 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1284/2614] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:43.236283 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b828d8-3f1f-4f19-86b5-46eff95f58c5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:43.246917 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b828d8-3f1f-4f19-86b5-46eff95f58c5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:43.251796 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8840df8-ecc2-4193-8f3a-3c3ae147dca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:43.252389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8840df8-ecc2-4193-8f3a-3c3ae147dca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:43.252667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8840df8-ecc2-4193-8f3a-3c3ae147dca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:43.252969 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8840df8-ecc2-4193-8f3a-3c3ae147dca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:43.253605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8840df8-ecc2-4193-8f3a-3c3ae147dca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:43.255189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8840df8-ecc2-4193-8f3a-3c3ae147dca7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:43.266869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1331/2615] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:43.271531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0807dd-5b52-48d6-97fb-0b0a1c431486 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:43.283704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0807dd-5b52-48d6-97fb-0b0a1c431486 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:43.288432 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc72b8-346d-466b-9469-60ee41e40623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:43.297513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc72b8-346d-466b-9469-60ee41e40623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:43.297513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc72b8-346d-466b-9469-60ee41e40623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:43.297513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc72b8-346d-466b-9469-60ee41e40623 admin admin] PATH_INFO: `/v3/projects/7a3ebe5cd4924ff19ddce156e9ac0a47/users/43ea575bacf944c5808c9f1d74988f07/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:43.306419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc72b8-346d-466b-9469-60ee41e40623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a3ebe5cd4924ff19ddce156e9ac0a47, user_id=43ea575bacf944c5808c9f1d74988f07, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:43.310716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc72b8-346d-466b-9469-60ee41e40623 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:43.414186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1285/2616] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:43 2022] PUT /identity/v3/projects/7a3ebe5cd4924ff19ddce156e9ac0a47/users/43ea575bacf944c5808c9f1d74988f07/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:43.426825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e99e9-7086-48cc-b8e1-dbe556c6658e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:43.427113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e99e9-7086-48cc-b8e1-dbe556c6658e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:43.427503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e99e9-7086-48cc-b8e1-dbe556c6658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:43.461470 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-6f0e99e9-7086-48cc-b8e1-dbe556c6658e None None] MFA Rules not processed for user `43ea575bacf944c5808c9f1d74988f07`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:43.462668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6f0e99e9-7086-48cc-b8e1-dbe556c6658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:43.540964 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1332/2617] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:43.545808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc72b8-346d-466b-9469-60ee41e40623 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:43.623445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:43.631050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc72b8-346d-466b-9469-60ee41e40623 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:43.637749 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cf94e-18b0-45e0-b87e-fe342b7caa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:43.638512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cf94e-18b0-45e0-b87e-fe342b7caa8f admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:43.638847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cf94e-18b0-45e0-b87e-fe342b7caa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:43.639190 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cf94e-18b0-45e0-b87e-fe342b7caa8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:43.639942 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cf94e-18b0-45e0-b87e-fe342b7caa8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:43.642026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cf94e-18b0-45e0-b87e-fe342b7caa8f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:43.715630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1286/2618] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:43 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:43.719740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e99e9-7086-48cc-b8e1-dbe556c6658e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:43.731351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e99e9-7086-48cc-b8e1-dbe556c6658e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:43.743697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:43.787664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6334a-6288-4181-8853-17d89c8f1ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:43.788301 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6334a-6288-4181-8853-17d89c8f1ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:43.788576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6334a-6288-4181-8853-17d89c8f1ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:43.788874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6334a-6288-4181-8853-17d89c8f1ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:43.789515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6334a-6288-4181-8853-17d89c8f1ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:43.791434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6334a-6288-4181-8853-17d89c8f1ff9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:43.873862 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1333/2619] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:43 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:43.877829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cf94e-18b0-45e0-b87e-fe342b7caa8f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:43.904313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cf94e-18b0-45e0-b87e-fe342b7caa8f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:43.910208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07a6bd-f1b9-4986-8e74-c7bc056f8558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:43.911183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a6bd-f1b9-4986-8e74-c7bc056f8558 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:43.912556 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a6bd-f1b9-4986-8e74-c7bc056f8558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:43.913073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a6bd-f1b9-4986-8e74-c7bc056f8558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:43.914154 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07a6bd-f1b9-4986-8e74-c7bc056f8558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:43.920564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07a6bd-f1b9-4986-8e74-c7bc056f8558 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:43.945761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1287/2620] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:43.955642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6334a-6288-4181-8853-17d89c8f1ff9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:43.969108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6334a-6288-4181-8853-17d89c8f1ff9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:43.978827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fc026-1d53-4392-9eac-5c528a5c0a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:43.980429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fc026-1d53-4392-9eac-5c528a5c0a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:43.981016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fc026-1d53-4392-9eac-5c528a5c0a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:43.981419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fc026-1d53-4392-9eac-5c528a5c0a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:43.982265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fc026-1d53-4392-9eac-5c528a5c0a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:43.983996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fc026-1d53-4392-9eac-5c528a5c0a12 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:43.996372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1334/2621] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:44.001232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07a6bd-f1b9-4986-8e74-c7bc056f8558 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:44.020676 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07a6bd-f1b9-4986-8e74-c7bc056f8558 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:44.027303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54736ed1-a101-42e5-9172-7f6b17cae38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:44.028643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54736ed1-a101-42e5-9172-7f6b17cae38d admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:44.028979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54736ed1-a101-42e5-9172-7f6b17cae38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:44.029225 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54736ed1-a101-42e5-9172-7f6b17cae38d admin admin] PATH_INFO: `/v3/projects/108762798fb648b0a3b146feaa3667eb/users/42eed51989364b6083fa1c47c4f33fb3/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:44.051554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54736ed1-a101-42e5-9172-7f6b17cae38d admin admin] RBAC: Authorizing `identity:create_grant(project_id=108762798fb648b0a3b146feaa3667eb, user_id=42eed51989364b6083fa1c47c4f33fb3, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:44.053945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54736ed1-a101-42e5-9172-7f6b17cae38d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:44.177087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1288/2622] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:43 2022] PUT /identity/v3/projects/108762798fb648b0a3b146feaa3667eb/users/42eed51989364b6083fa1c47c4f33fb3/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:44.187913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f32ef-2186-48cb-a6d0-911e1876b4c7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:44.188205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f32ef-2186-48cb-a6d0-911e1876b4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:44.188491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f32ef-2186-48cb-a6d0-911e1876b4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:44.231348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b76f32ef-2186-48cb-a6d0-911e1876b4c7 None None] MFA Rules not processed for user `42eed51989364b6083fa1c47c4f33fb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:44.231348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b76f32ef-2186-48cb-a6d0-911e1876b4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:44.315267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1335/2623] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:44.320038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54736ed1-a101-42e5-9172-7f6b17cae38d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:44.404542 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54736ed1-a101-42e5-9172-7f6b17cae38d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:44.409437 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b60441-52c9-49a5-91b5-5cf840997b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:44.414744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b60441-52c9-49a5-91b5-5cf840997b44 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:44.414744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b60441-52c9-49a5-91b5-5cf840997b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:44.414744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b60441-52c9-49a5-91b5-5cf840997b44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:44.414744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b60441-52c9-49a5-91b5-5cf840997b44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:44.414744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b60441-52c9-49a5-91b5-5cf840997b44 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:44.521393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1289/2624] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:44 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:44.525674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f32ef-2186-48cb-a6d0-911e1876b4c7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:44.537328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f32ef-2186-48cb-a6d0-911e1876b4c7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:44.614892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b245d-0cf1-4cfc-859d-860e3b59f446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:44.615504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b245d-0cf1-4cfc-859d-860e3b59f446 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:44.615801 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b245d-0cf1-4cfc-859d-860e3b59f446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:44.616071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b245d-0cf1-4cfc-859d-860e3b59f446 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:44.616712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b245d-0cf1-4cfc-859d-860e3b59f446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:44.618826 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b245d-0cf1-4cfc-859d-860e3b59f446 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:44.717086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1336/2625] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:44 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:44.723163 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b60441-52c9-49a5-91b5-5cf840997b44 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:44.740912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b60441-52c9-49a5-91b5-5cf840997b44 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:44.749142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cafe8e-4ec1-4588-b358-d22aa902cf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:44.750149 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cafe8e-4ec1-4588-b358-d22aa902cf92 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:44.750632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cafe8e-4ec1-4588-b358-d22aa902cf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:44.751357 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cafe8e-4ec1-4588-b358-d22aa902cf92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:44.752460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cafe8e-4ec1-4588-b358-d22aa902cf92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:44.754309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cafe8e-4ec1-4588-b358-d22aa902cf92 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:44.766498 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1290/2626] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:44.771069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b245d-0cf1-4cfc-859d-860e3b59f446 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:44.783554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b245d-0cf1-4cfc-859d-860e3b59f446 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:44.789138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8c5f9-1f35-4dae-b69c-541291eb76eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:44.789821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8c5f9-1f35-4dae-b69c-541291eb76eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:44.793924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8c5f9-1f35-4dae-b69c-541291eb76eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:44.793924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8c5f9-1f35-4dae-b69c-541291eb76eb admin admin] PATH_INFO: `/v3/projects/aa4cf87e2db44780b5ad9885c3922020/users/4544945742084a448dc0fc6df46573d2/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:44.817168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:44.821856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8c5f9-1f35-4dae-b69c-541291eb76eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4cf87e2db44780b5ad9885c3922020, user_id=4544945742084a448dc0fc6df46573d2, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:44.825597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8c5f9-1f35-4dae-b69c-541291eb76eb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:44.903265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1337/2627] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:44 2022] PUT /identity/v3/projects/aa4cf87e2db44780b5ad9885c3922020/users/4544945742084a448dc0fc6df46573d2/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:44.907782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cafe8e-4ec1-4588-b358-d22aa902cf92 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:44.918795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cafe8e-4ec1-4588-b358-d22aa902cf92 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:44.998573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333f505-fec4-4470-b994-e3d5a2135420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:44.999427 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333f505-fec4-4470-b994-e3d5a2135420 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:44.999704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333f505-fec4-4470-b994-e3d5a2135420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:44.999972 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333f505-fec4-4470-b994-e3d5a2135420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:45.000622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8333f505-fec4-4470-b994-e3d5a2135420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:45.002502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8333f505-fec4-4470-b994-e3d5a2135420 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:45.014022 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1291/2628] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:45.018303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8c5f9-1f35-4dae-b69c-541291eb76eb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:45.029293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8c5f9-1f35-4dae-b69c-541291eb76eb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:45.036924 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72076-0340-4d51-a3e1-f5de0c879e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:45.037517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72076-0340-4d51-a3e1-f5de0c879e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:45.037790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72076-0340-4d51-a3e1-f5de0c879e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:45.038058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72076-0340-4d51-a3e1-f5de0c879e0c admin admin] PATH_INFO: `/v3/projects/aa4cf87e2db44780b5ad9885c3922020/users/4544945742084a448dc0fc6df46573d2/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:45.039996 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d72076-0340-4d51-a3e1-f5de0c879e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4cf87e2db44780b5ad9885c3922020, user_id=4544945742084a448dc0fc6df46573d2, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:45.042928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d72076-0340-4d51-a3e1-f5de0c879e0c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:45.054203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1338/2629] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:45 2022] PUT /identity/v3/projects/aa4cf87e2db44780b5ad9885c3922020/users/4544945742084a448dc0fc6df46573d2/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:45.066079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8289669-b289-4f51-bc68-8c7bfbbb910a None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:45.066358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8289669-b289-4f51-bc68-8c7bfbbb910a None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:45.066649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8289669-b289-4f51-bc68-8c7bfbbb910a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:45.098903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-f8289669-b289-4f51-bc68-8c7bfbbb910a None None] MFA Rules not processed for user `4544945742084a448dc0fc6df46573d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:45.100191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f8289669-b289-4f51-bc68-8c7bfbbb910a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:45.105414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:45.202245 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1292/2630] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:45.213588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72076-0340-4d51-a3e1-f5de0c879e0c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:45.215442 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-16d72076-0340-4d51-a3e1-f5de0c879e0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:45.275689 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8289669-b289-4f51-bc68-8c7bfbbb910a None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:45.277716 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f8289669-b289-4f51-bc68-8c7bfbbb910a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:45.280316 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72076-0340-4d51-a3e1-f5de0c879e0c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:45.285949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bf5e4-9e28-49b9-b814-228db0b27493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:45.286877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bf5e4-9e28-49b9-b814-228db0b27493 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:45.287377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bf5e4-9e28-49b9-b814-228db0b27493 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:45.287642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54bf5e4-9e28-49b9-b814-228db0b27493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:45.299023 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bf5e4-9e28-49b9-b814-228db0b27493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:45.300478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54bf5e4-9e28-49b9-b814-228db0b27493 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:45.312186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1339/2631] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:45.355768 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8289669-b289-4f51-bc68-8c7bfbbb910a None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:45.361929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:45.362560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:45.362836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:45.363108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] PATH_INFO: `/v3/users/d6c0d6aa73754070b951b9c9f0294682` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:45.364223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6c0d6aa73754070b951b9c9f0294682)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:45.366158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:45.441815 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bf5e4-9e28-49b9-b814-228db0b27493 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:45.443194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b54bf5e4-9e28-49b9-b814-228db0b27493 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:45.454778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54bf5e4-9e28-49b9-b814-228db0b27493 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:45.459002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c6fb4b-cf51-4064-8504-788f2ad93557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:45.459599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6fb4b-cf51-4064-8504-788f2ad93557 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:45.459866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6fb4b-cf51-4064-8504-788f2ad93557 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:45.460131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c6fb4b-cf51-4064-8504-788f2ad93557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:45.528074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6fb4b-cf51-4064-8504-788f2ad93557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:45.529544 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c6fb4b-cf51-4064-8504-788f2ad93557 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:45.538578 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1340/2632] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:45.579541 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6c0d6aa73754070b951b9c9f0294682'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:45.580487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6c0d6aa73754070b951b9c9f0294682'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:45.581159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6c0d6aa73754070b951b9c9f0294682'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:45.588642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6c0d6aa73754070b951b9c9f0294682'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:45.601596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6c0d6aa73754070b951b9c9f0294682'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:45.678757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1293/2633] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:45 2022] DELETE /identity/v3/users/d6c0d6aa73754070b951b9c9f0294682 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:45.683662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca9330-42be-474a-be32-71a8831d5a05 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:45.684043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca9330-42be-474a-be32-71a8831d5a05 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:45.684291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca9330-42be-474a-be32-71a8831d5a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:45.704667 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d1ca9330-42be-474a-be32-71a8831d5a05 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:45.705399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d1ca9330-42be-474a-be32-71a8831d5a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:45.769151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1341/2634] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:45.780368 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:45.782172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:45.805672 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:45.846273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a70be-8fa1-49d3-8606-17e9a72d2675 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:45.849889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15537f-8d91-4de4-b2ec-5e168100fdeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:45.850494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15537f-8d91-4de4-b2ec-5e168100fdeb service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:45.850784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15537f-8d91-4de4-b2ec-5e168100fdeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:45.851236 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15537f-8d91-4de4-b2ec-5e168100fdeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:45.865696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f15537f-8d91-4de4-b2ec-5e168100fdeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:45.867620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f15537f-8d91-4de4-b2ec-5e168100fdeb service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:45.884519 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1294/2635] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:46.179396 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca9330-42be-474a-be32-71a8831d5a05 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:46.192719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d1ca9330-42be-474a-be32-71a8831d5a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:46.215251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca9330-42be-474a-be32-71a8831d5a05 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:46.226781 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:46.228779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:46.229091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:46.229525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] PATH_INFO: `/v3/users/c814f5342283472aab1a4d0b71f3f6c3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:46.231291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c814f5342283472aab1a4d0b71f3f6c3)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:46.233498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:46.253767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:46.398576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15537f-8d91-4de4-b2ec-5e168100fdeb service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:46.400322 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9f15537f-8d91-4de4-b2ec-5e168100fdeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:46.428166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15537f-8d91-4de4-b2ec-5e168100fdeb service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:46.435854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:46.436525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:46.436823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:46.437125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] PATH_INFO: `/v3/projects/4624cfd084b543b592d34f7064cc3acb` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:46.438430 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4624cfd084b543b592d34f7064cc3acb)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:46.440373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:46.478591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c814f5342283472aab1a4d0b71f3f6c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.564517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c814f5342283472aab1a4d0b71f3f6c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.587264 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c814f5342283472aab1a4d0b71f3f6c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.588101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c814f5342283472aab1a4d0b71f3f6c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.595459 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cd2ec733-ec96-4d5b-88be-fceebe099478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c814f5342283472aab1a4d0b71f3f6c3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.603540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1342/2636] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:46 2022] DELETE /identity/v3/users/c814f5342283472aab1a4d0b71f3f6c3 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:46.608674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306a897-d347-4e34-92a9-b29fcd9b0126 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:46.609825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306a897-d347-4e34-92a9-b29fcd9b0126 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:46.610271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306a897-d347-4e34-92a9-b29fcd9b0126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:46.627955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4624cfd084b543b592d34f7064cc3acb', 'request_id': 'req-6c10d15a-7788-4942-bb7e-2121bc21ed20'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.633971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4624cfd084b543b592d34f7064cc3acb', 'request_id': 'req-6c10d15a-7788-4942-bb7e-2121bc21ed20'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.637720 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f306a897-d347-4e34-92a9-b29fcd9b0126 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:46.639051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f306a897-d347-4e34-92a9-b29fcd9b0126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:46.701656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1343/2637] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:46.711696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306a897-d347-4e34-92a9-b29fcd9b0126 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:46.713121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f306a897-d347-4e34-92a9-b29fcd9b0126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:46.732772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4624cfd084b543b592d34f7064cc3acb', 'request_id': 'req-6c10d15a-7788-4942-bb7e-2121bc21ed20'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.746887 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] The token cache is being invalidate because project 4624cfd084b543b592d34f7064cc3acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:46.747452 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.748148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:46.749871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1295/2638] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:46 2022] DELETE /identity/v3/projects/4624cfd084b543b592d34f7064cc3acb => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:46.754104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:46.755506 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:46.768603 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10d15a-7788-4942-bb7e-2121bc21ed20 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:46.773823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:46.774464 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:46.774736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:46.775007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] PATH_INFO: `/v3/users/c671e5cd1b6e40e5b760c7def8844f58` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:46.776158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c671e5cd1b6e40e5b760c7def8844f58)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:46.778428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:46.785051 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306a897-d347-4e34-92a9-b29fcd9b0126 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:46.790269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:46.790269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:46.790770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:46.791070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:46.793934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:46.809956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:46.811405 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:46.822697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1344/2639] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:47.010526 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c671e5cd1b6e40e5b760c7def8844f58'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.011235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c671e5cd1b6e40e5b760c7def8844f58'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.011854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c671e5cd1b6e40e5b760c7def8844f58'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.019330 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c671e5cd1b6e40e5b760c7def8844f58'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.032276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c671e5cd1b6e40e5b760c7def8844f58'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.149233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1296/2640] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:46 2022] DELETE /identity/v3/users/c671e5cd1b6e40e5b760c7def8844f58 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:47.286468 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:47.288067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:47.308115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f301c7-7a03-4fa3-9a76-939efb4597d3 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:47.347596 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:47.372761 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:47.373420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:47.373694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:47.374071 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] PATH_INFO: `/v3/projects/45a0e67cefb64a8983aa4220e75cf370` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:47.375095 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a0e67cefb64a8983aa4220e75cf370)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:47.378067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:47.550575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a0e67cefb64a8983aa4220e75cf370', 'request_id': 'req-74e2395c-c097-4cbb-901b-f3fc859c6efa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.557194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a0e67cefb64a8983aa4220e75cf370', 'request_id': 'req-74e2395c-c097-4cbb-901b-f3fc859c6efa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.629912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a0e67cefb64a8983aa4220e75cf370', 'request_id': 'req-74e2395c-c097-4cbb-901b-f3fc859c6efa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.642323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] The token cache is being invalidate because project 45a0e67cefb64a8983aa4220e75cf370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:47.642603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.643249 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.644773 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1345/2641] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:47 2022] DELETE /identity/v3/projects/45a0e67cefb64a8983aa4220e75cf370 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:47.648958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:47.651112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:47.665480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835653b6-d3b7-4b04-b0c1-7f506a72596a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:47.678670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:47.686584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:47.688155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:47.729776 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:47.730830 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:47.731218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:47.731561 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] PATH_INFO: `/v3/users/ee2d7ee7dfcf4183b47dd2760201ba31` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:47.732764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2d7ee7dfcf4183b47dd2760201ba31)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:47.735005 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:47.767593 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2395c-c097-4cbb-901b-f3fc859c6efa admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:47.772198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:47.772851 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:47.773130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:47.773400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] PATH_INFO: `/v3/projects/e77cec56c8bc4e488b0ea8ab2e6b2e14` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:47.774515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e77cec56c8bc4e488b0ea8ab2e6b2e14)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:47.776683 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:47.943436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77cec56c8bc4e488b0ea8ab2e6b2e14', 'request_id': 'req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.948172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2d7ee7dfcf4183b47dd2760201ba31'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.949100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2d7ee7dfcf4183b47dd2760201ba31'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.950113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2d7ee7dfcf4183b47dd2760201ba31'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.953835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77cec56c8bc4e488b0ea8ab2e6b2e14', 'request_id': 'req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.961797 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2d7ee7dfcf4183b47dd2760201ba31'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:47.980176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-05fe7325-487d-4f3b-8a6f-55403530d004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2d7ee7dfcf4183b47dd2760201ba31'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:48.041174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77cec56c8bc4e488b0ea8ab2e6b2e14', 'request_id': 'req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:48.047028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:48.053067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] The token cache is being invalidate because project e77cec56c8bc4e488b0ea8ab2e6b2e14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:48.053350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:48.053981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-730835f6-8a2f-47fc-aaef-e40ad36e2ce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:48.055518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1346/2642] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:47 2022] DELETE /identity/v3/projects/e77cec56c8bc4e488b0ea8ab2e6b2e14 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:48.060277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dd6d4-56ff-4140-b0be-4c065c8c263c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.060550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dd6d4-56ff-4140-b0be-4c065c8c263c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.060842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dd6d4-56ff-4140-b0be-4c065c8c263c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.080647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-784dd6d4-56ff-4140-b0be-4c065c8c263c None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:48.085645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-784dd6d4-56ff-4140-b0be-4c065c8c263c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:48.138336 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1297/2643] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:47 2022] DELETE /identity/v3/users/ee2d7ee7dfcf4183b47dd2760201ba31 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:48.148422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1347/2644] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:48.156860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3c7dd-3fa9-453c-b3a5-696a207aaa89 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.157272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3c7dd-3fa9-453c-b3a5-696a207aaa89 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.157625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3c7dd-3fa9-453c-b3a5-696a207aaa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.177940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a1d3c7dd-3fa9-453c-b3a5-696a207aaa89 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:48.179148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a1d3c7dd-3fa9-453c-b3a5-696a207aaa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:48.243292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1298/2645] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:48.248120 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dd6d4-56ff-4140-b0be-4c065c8c263c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:48.258605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dd6d4-56ff-4140-b0be-4c065c8c263c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:48.264497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdca7c2-171f-49d7-b1e8-367a7a67215c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:48.265131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdca7c2-171f-49d7-b1e8-367a7a67215c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.265416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdca7c2-171f-49d7-b1e8-367a7a67215c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.265685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdca7c2-171f-49d7-b1e8-367a7a67215c admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.266335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdca7c2-171f-49d7-b1e8-367a7a67215c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:48.266636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdca7c2-171f-49d7-b1e8-367a7a67215c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:48.268261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdca7c2-171f-49d7-b1e8-367a7a67215c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:48.281625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1348/2646] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:46:48.286037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3c7dd-3fa9-453c-b3a5-696a207aaa89 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:48.296897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3c7dd-3fa9-453c-b3a5-696a207aaa89 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:48.301512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464dce58-25b2-40c6-9f02-e3f22f0852f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:48.302119 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dce58-25b2-40c6-9f02-e3f22f0852f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.302442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dce58-25b2-40c6-9f02-e3f22f0852f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.302729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dce58-25b2-40c6-9f02-e3f22f0852f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.303388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464dce58-25b2-40c6-9f02-e3f22f0852f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:48.305334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464dce58-25b2-40c6-9f02-e3f22f0852f9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:48.393025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1299/2647] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:48 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:48.399724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdca7c2-171f-49d7-b1e8-367a7a67215c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:48.411587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdca7c2-171f-49d7-b1e8-367a7a67215c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:48.480057 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b3507-998b-4f9b-8108-9010ccd732fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:48.481366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b3507-998b-4f9b-8108-9010ccd732fb admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.482122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b3507-998b-4f9b-8108-9010ccd732fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.482122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b3507-998b-4f9b-8108-9010ccd732fb admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.482557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b3507-998b-4f9b-8108-9010ccd732fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:48.484474 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b3507-998b-4f9b-8108-9010ccd732fb admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:48.627466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464dce58-25b2-40c6-9f02-e3f22f0852f9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:48.633629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-464dce58-25b2-40c6-9f02-e3f22f0852f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:48.647552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464dce58-25b2-40c6-9f02-e3f22f0852f9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:48.648393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1349/2648] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:48 2022] POST /identity/v3/users => generated 301 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:48.650332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b3507-998b-4f9b-8108-9010ccd732fb admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:48.653107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:48.654719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.655123 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.655380 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] PATH_INFO: `/v3/projects/25261c0c5fc14e2d8d4095efe3f93c18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.656639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25261c0c5fc14e2d8d4095efe3f93c18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:48.658850 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:48.663141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b3507-998b-4f9b-8108-9010ccd732fb admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:48.669168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088be6d-21bd-4592-ad3c-17dd2500051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:48.670821 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088be6d-21bd-4592-ad3c-17dd2500051c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.671269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088be6d-21bd-4592-ad3c-17dd2500051c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.671660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088be6d-21bd-4592-ad3c-17dd2500051c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.672648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088be6d-21bd-4592-ad3c-17dd2500051c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:48.674964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:48.675484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088be6d-21bd-4592-ad3c-17dd2500051c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:48.688331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1350/2649] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:48.692274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088be6d-21bd-4592-ad3c-17dd2500051c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:48.703258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088be6d-21bd-4592-ad3c-17dd2500051c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:48.707943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dc322-127a-4678-983d-74105e7acd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:48.708536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dc322-127a-4678-983d-74105e7acd8d admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.708835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dc322-127a-4678-983d-74105e7acd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.709110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726dc322-127a-4678-983d-74105e7acd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.709743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dc322-127a-4678-983d-74105e7acd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:48.711328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726dc322-127a-4678-983d-74105e7acd8d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:48.722387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1351/2650] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:48.726607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dc322-127a-4678-983d-74105e7acd8d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:48.737485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726dc322-127a-4678-983d-74105e7acd8d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:48.742393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b948f9-4766-4db0-b698-cdb7ebe404f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:48.743031 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b948f9-4766-4db0-b698-cdb7ebe404f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.743304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b948f9-4766-4db0-b698-cdb7ebe404f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.743575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b948f9-4766-4db0-b698-cdb7ebe404f5 admin admin] PATH_INFO: `/v3/projects/2ec1ed87aa724146976ca73ef05f48a3/users/d7fbb661dc8c4b66a4916ec380582267/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.761409 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b948f9-4766-4db0-b698-cdb7ebe404f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec1ed87aa724146976ca73ef05f48a3, user_id=d7fbb661dc8c4b66a4916ec380582267, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:48.763956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b948f9-4766-4db0-b698-cdb7ebe404f5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:48.858607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1352/2651] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:48 2022] PUT /identity/v3/projects/2ec1ed87aa724146976ca73ef05f48a3/users/d7fbb661dc8c4b66a4916ec380582267/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:48.866955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25261c0c5fc14e2d8d4095efe3f93c18', 'request_id': 'req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:48.870675 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7d4d0-7d4a-45cc-8597-c948e0684ae3 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:48.870886 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7d4d0-7d4a-45cc-8597-c948e0684ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:48.871090 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7d4d0-7d4a-45cc-8597-c948e0684ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:48.873482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25261c0c5fc14e2d8d4095efe3f93c18', 'request_id': 'req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:48.899625 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f2c7d4d0-7d4a-45cc-8597-c948e0684ae3 None None] MFA Rules not processed for user `d7fbb661dc8c4b66a4916ec380582267`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:48.900838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f2c7d4d0-7d4a-45cc-8597-c948e0684ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:48.913217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:48.972127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1353/2652] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:48 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:48.976702 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7d4d0-7d4a-45cc-8597-c948e0684ae3 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:48.996152 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25261c0c5fc14e2d8d4095efe3f93c18', 'request_id': 'req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:49.008870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] The token cache is being invalidate because project 25261c0c5fc14e2d8d4095efe3f93c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:49.008870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:49.009364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-e80a72bf-a7b0-4b56-b6fc-4415206b8f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:49.011394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1300/2653] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:48 2022] DELETE /identity/v3/projects/25261c0c5fc14e2d8d4095efe3f93c18 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:49.015782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22a1e2-990e-4c98-a302-d9c4ff2fb2dc None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.016046 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22a1e2-990e-4c98-a302-d9c4ff2fb2dc None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.016348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22a1e2-990e-4c98-a302-d9c4ff2fb2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.041048 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-fa22a1e2-990e-4c98-a302-d9c4ff2fb2dc None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:49.043118 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fa22a1e2-990e-4c98-a302-d9c4ff2fb2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:49.047351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7d4d0-7d4a-45cc-8597-c948e0684ae3 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.052174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1301/2654] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:49.063661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e4bdb-c188-45bd-8d79-43753185a844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.063661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e4bdb-c188-45bd-8d79-43753185a844 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.063661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e4bdb-c188-45bd-8d79-43753185a844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.063661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e4bdb-c188-45bd-8d79-43753185a844 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.064102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e4bdb-c188-45bd-8d79-43753185a844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.064102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e4bdb-c188-45bd-8d79-43753185a844 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.065632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7e265-51f7-41f5-9889-87d390da8901 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.066010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7e265-51f7-41f5-9889-87d390da8901 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.066385 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7e265-51f7-41f5-9889-87d390da8901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.086797 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-64c7e265-51f7-41f5-9889-87d390da8901 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:49.087986 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-64c7e265-51f7-41f5-9889-87d390da8901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:49.096473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1302/2655] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:49.100602 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7e265-51f7-41f5-9889-87d390da8901 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.110323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7e265-51f7-41f5-9889-87d390da8901 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.114873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0f709-63f3-4d18-a1be-ac69cf32af6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.115467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0f709-63f3-4d18-a1be-ac69cf32af6f admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.115744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0f709-63f3-4d18-a1be-ac69cf32af6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.116007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0f709-63f3-4d18-a1be-ac69cf32af6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.116650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0f709-63f3-4d18-a1be-ac69cf32af6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:49.116965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0f709-63f3-4d18-a1be-ac69cf32af6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.118529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0f709-63f3-4d18-a1be-ac69cf32af6f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.131045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1303/2656] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:46:49.135173 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0f709-63f3-4d18-a1be-ac69cf32af6f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.144872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0f709-63f3-4d18-a1be-ac69cf32af6f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.149360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238960fa-1502-4dc8-8c98-c55ffeee6acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.149936 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238960fa-1502-4dc8-8c98-c55ffeee6acf admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.150209 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238960fa-1502-4dc8-8c98-c55ffeee6acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.150475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238960fa-1502-4dc8-8c98-c55ffeee6acf admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.151076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238960fa-1502-4dc8-8c98-c55ffeee6acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.153020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238960fa-1502-4dc8-8c98-c55ffeee6acf admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.196532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1354/2657] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:48 2022] POST /identity/v3/projects => generated 401 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:49.201502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e4bdb-c188-45bd-8d79-43753185a844 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.203484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-457e4bdb-c188-45bd-8d79-43753185a844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:49.262812 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1304/2658] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:49 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:49.263939 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238960fa-1502-4dc8-8c98-c55ffeee6acf admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.281734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e4bdb-c188-45bd-8d79-43753185a844 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.345417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:49.349232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238960fa-1502-4dc8-8c98-c55ffeee6acf admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.351812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6eaf2-6311-4441-9b46-eb836c6fd79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.352922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6eaf2-6311-4441-9b46-eb836c6fd79e admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.353205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6eaf2-6311-4441-9b46-eb836c6fd79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.353478 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6eaf2-6311-4441-9b46-eb836c6fd79e admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.354100 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6eaf2-6311-4441-9b46-eb836c6fd79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.355582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907893d8-42e2-41ed-aec2-ce258e2dc3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.356071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907893d8-42e2-41ed-aec2-ce258e2dc3ca admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.356354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907893d8-42e2-41ed-aec2-ce258e2dc3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.356621 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6eaf2-6311-4441-9b46-eb836c6fd79e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.356763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907893d8-42e2-41ed-aec2-ce258e2dc3ca admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.360639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907893d8-42e2-41ed-aec2-ce258e2dc3ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.360639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907893d8-42e2-41ed-aec2-ce258e2dc3ca admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.441326 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1355/2659] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:49 2022] POST /identity/v3/users => generated 307 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:49.445679 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6eaf2-6311-4441-9b46-eb836c6fd79e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.455709 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6eaf2-6311-4441-9b46-eb836c6fd79e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.465133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1305/2660] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:49 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:49.466029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c696806-d780-4dc7-b893-5fc7686e83ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.467668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c696806-d780-4dc7-b893-5fc7686e83ff admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.467668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c696806-d780-4dc7-b893-5fc7686e83ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.467668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c696806-d780-4dc7-b893-5fc7686e83ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.468002 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c696806-d780-4dc7-b893-5fc7686e83ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.469900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c696806-d780-4dc7-b893-5fc7686e83ff admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.470632 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907893d8-42e2-41ed-aec2-ce258e2dc3ca admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.482032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907893d8-42e2-41ed-aec2-ce258e2dc3ca admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.482269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1356/2661] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:49.486713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c696806-d780-4dc7-b893-5fc7686e83ff admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.489973 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf514b9-0574-4571-b834-b760119567f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.490690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf514b9-0574-4571-b834-b760119567f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.491062 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf514b9-0574-4571-b834-b760119567f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.491859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf514b9-0574-4571-b834-b760119567f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.492447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf514b9-0574-4571-b834-b760119567f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.494574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf514b9-0574-4571-b834-b760119567f2 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.497626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c696806-d780-4dc7-b893-5fc7686e83ff admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.502706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4d53-0f9f-40c6-8374-52e62c8242e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.504362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4d53-0f9f-40c6-8374-52e62c8242e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.504362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4d53-0f9f-40c6-8374-52e62c8242e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.504362 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4d53-0f9f-40c6-8374-52e62c8242e0 admin admin] PATH_INFO: `/v3/projects/fc498272f2194598b18758ed04694331/users/6bbaa71b038e4256aef0ad0d453d08b1/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.509903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1306/2662] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:49.517682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf514b9-0574-4571-b834-b760119567f2 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.529358 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf514b9-0574-4571-b834-b760119567f2 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.530836 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b4d53-0f9f-40c6-8374-52e62c8242e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc498272f2194598b18758ed04694331, user_id=6bbaa71b038e4256aef0ad0d453d08b1, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.533239 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b4d53-0f9f-40c6-8374-52e62c8242e0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.539512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5304d5e-3420-4553-9225-8540efa815f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.542863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5304d5e-3420-4553-9225-8540efa815f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.542863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5304d5e-3420-4553-9225-8540efa815f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.542863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5304d5e-3420-4553-9225-8540efa815f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.542863 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5304d5e-3420-4553-9225-8540efa815f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.544422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5304d5e-3420-4553-9225-8540efa815f0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.556183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1307/2663] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:49.560348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5304d5e-3420-4553-9225-8540efa815f0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.570540 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5304d5e-3420-4553-9225-8540efa815f0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.575712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e39bc-85d5-4a52-b277-9775451e4567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.576365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e39bc-85d5-4a52-b277-9775451e4567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.576640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e39bc-85d5-4a52-b277-9775451e4567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.576945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e39bc-85d5-4a52-b277-9775451e4567 admin admin] PATH_INFO: `/v3/projects/efe2ef9f238548b7a12d56888fc8ceaa/users/33521fb407174bf6b60f65c1a5c25c63/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.592858 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e39bc-85d5-4a52-b277-9775451e4567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe2ef9f238548b7a12d56888fc8ceaa, user_id=33521fb407174bf6b60f65c1a5c25c63, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.595545 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e39bc-85d5-4a52-b277-9775451e4567 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.597791 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:49.630584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1357/2664] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:49 2022] PUT /identity/v3/projects/fc498272f2194598b18758ed04694331/users/6bbaa71b038e4256aef0ad0d453d08b1/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:49.634609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4d53-0f9f-40c6-8374-52e62c8242e0 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.670892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1308/2665] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:49 2022] PUT /identity/v3/projects/efe2ef9f238548b7a12d56888fc8ceaa/users/33521fb407174bf6b60f65c1a5c25c63/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:49.682409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d69f-9783-4495-ba47-6c7e357ff534 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.682762 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d69f-9783-4495-ba47-6c7e357ff534 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.683208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d69f-9783-4495-ba47-6c7e357ff534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.707211 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4d53-0f9f-40c6-8374-52e62c8242e0 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.712704 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-9fb1d69f-9783-4495-ba47-6c7e357ff534 None None] MFA Rules not processed for user `33521fb407174bf6b60f65c1a5c25c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:49.714549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9fb1d69f-9783-4495-ba47-6c7e357ff534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:49.787131 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79adae1-1398-48b1-809c-035fdcdc7ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.787827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79adae1-1398-48b1-809c-035fdcdc7ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.788169 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79adae1-1398-48b1-809c-035fdcdc7ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.788572 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79adae1-1398-48b1-809c-035fdcdc7ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.789381 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79adae1-1398-48b1-809c-035fdcdc7ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.791084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79adae1-1398-48b1-809c-035fdcdc7ed7 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.799688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1309/2666] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:49.804914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1358/2667] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:49.806399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1d69f-9783-4495-ba47-6c7e357ff534 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.806603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79adae1-1398-48b1-809c-035fdcdc7ed7 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.816743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79adae1-1398-48b1-809c-035fdcdc7ed7 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.817283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1d69f-9783-4495-ba47-6c7e357ff534 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:49.821682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25050bef-046c-4b40-8ab6-1fc1d8e58024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.822025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae8576-e112-417e-bcfd-274ba88cb072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:49.822341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25050bef-046c-4b40-8ab6-1fc1d8e58024 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.822617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25050bef-046c-4b40-8ab6-1fc1d8e58024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.822720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae8576-e112-417e-bcfd-274ba88cb072 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.822891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25050bef-046c-4b40-8ab6-1fc1d8e58024 admin admin] PATH_INFO: `/v3/projects/fc498272f2194598b18758ed04694331/users/6bbaa71b038e4256aef0ad0d453d08b1/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.823010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae8576-e112-417e-bcfd-274ba88cb072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.823828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae8576-e112-417e-bcfd-274ba88cb072 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.824324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae8576-e112-417e-bcfd-274ba88cb072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.825758 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25050bef-046c-4b40-8ab6-1fc1d8e58024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc498272f2194598b18758ed04694331, user_id=6bbaa71b038e4256aef0ad0d453d08b1, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:49.831831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae8576-e112-417e-bcfd-274ba88cb072 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.832108 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25050bef-046c-4b40-8ab6-1fc1d8e58024 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:49.840335 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1359/2668] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:49 2022] PUT /identity/v3/projects/fc498272f2194598b18758ed04694331/users/6bbaa71b038e4256aef0ad0d453d08b1/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:49.852152 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2c0db-6059-4a96-9088-7e4aaa1b16ab None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:49.852571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2c0db-6059-4a96-9088-7e4aaa1b16ab None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:49.852904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2c0db-6059-4a96-9088-7e4aaa1b16ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:49.897012 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-75c2c0db-6059-4a96-9088-7e4aaa1b16ab None None] MFA Rules not processed for user `6bbaa71b038e4256aef0ad0d453d08b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:49.898366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-75c2c0db-6059-4a96-9088-7e4aaa1b16ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:49.924132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:49.931103 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1310/2669] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:49 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:49.939096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae8576-e112-417e-bcfd-274ba88cb072 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:49.990026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1360/2670] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:49 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:50.005201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2c0db-6059-4a96-9088-7e4aaa1b16ab None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:50.007829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-75c2c0db-6059-4a96-9088-7e4aaa1b16ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:50.018132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae8576-e112-417e-bcfd-274ba88cb072 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:50.024810 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b486e-8233-46fe-a6d9-17172723ab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:50.025834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b486e-8233-46fe-a6d9-17172723ab26 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:50.026273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b486e-8233-46fe-a6d9-17172723ab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:50.028008 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b486e-8233-46fe-a6d9-17172723ab26 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:50.029849 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b486e-8233-46fe-a6d9-17172723ab26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:50.032990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b486e-8233-46fe-a6d9-17172723ab26 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:50.079279 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2c0db-6059-4a96-9088-7e4aaa1b16ab None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:50.126470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1311/2671] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:49 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:50.130491 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b486e-8233-46fe-a6d9-17172723ab26 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:50.142404 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b486e-8233-46fe-a6d9-17172723ab26 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:50.149446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86600a11-cb34-4dbe-bc7d-dd3914913e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:50.150648 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86600a11-cb34-4dbe-bc7d-dd3914913e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:50.151260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86600a11-cb34-4dbe-bc7d-dd3914913e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:50.151662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86600a11-cb34-4dbe-bc7d-dd3914913e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:50.152535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86600a11-cb34-4dbe-bc7d-dd3914913e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:50.154378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86600a11-cb34-4dbe-bc7d-dd3914913e43 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:50.155837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a28bcf-a805-4a2f-adb8-2103ebe17c59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:50.156455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a28bcf-a805-4a2f-adb8-2103ebe17c59 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:50.156736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a28bcf-a805-4a2f-adb8-2103ebe17c59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:50.157429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a28bcf-a805-4a2f-adb8-2103ebe17c59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:50.170853 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1312/2672] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:50.171879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86600a11-cb34-4dbe-bc7d-dd3914913e43 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:50.181654 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86600a11-cb34-4dbe-bc7d-dd3914913e43 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:50.187250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95a830-d1f1-4e04-aa05-022e10967d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:50.187897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95a830-d1f1-4e04-aa05-022e10967d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:50.188167 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95a830-d1f1-4e04-aa05-022e10967d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:50.188439 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95a830-d1f1-4e04-aa05-022e10967d81 admin admin] PATH_INFO: `/v3/projects/119c156e97d94625ad19d671452d7727/users/e46c69248d374c3bb9b4f6a73143de33/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:50.206480 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95a830-d1f1-4e04-aa05-022e10967d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=119c156e97d94625ad19d671452d7727, user_id=e46c69248d374c3bb9b4f6a73143de33, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:50.209178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95a830-d1f1-4e04-aa05-022e10967d81 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:50.231700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a28bcf-a805-4a2f-adb8-2103ebe17c59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:50.233302 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a28bcf-a805-4a2f-adb8-2103ebe17c59 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:50.246676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1361/2673] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:50 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:50.325037 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1313/2674] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:50 2022] PUT /identity/v3/projects/119c156e97d94625ad19d671452d7727/users/e46c69248d374c3bb9b4f6a73143de33/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:50.329273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a28bcf-a805-4a2f-adb8-2103ebe17c59 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:50.339797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a28bcf-a805-4a2f-adb8-2103ebe17c59 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:50.374513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:50.414984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aff7ae-fbf6-4ad4-b8ea-0b7e3c8f4a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:50.415688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aff7ae-fbf6-4ad4-b8ea-0b7e3c8f4a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:50.415962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aff7ae-fbf6-4ad4-b8ea-0b7e3c8f4a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:50.416232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aff7ae-fbf6-4ad4-b8ea-0b7e3c8f4a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:50.417060 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aff7ae-fbf6-4ad4-b8ea-0b7e3c8f4a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:50.418754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aff7ae-fbf6-4ad4-b8ea-0b7e3c8f4a00 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:50.428900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1362/2675] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:50.433720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95a830-d1f1-4e04-aa05-022e10967d81 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:50.444503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95a830-d1f1-4e04-aa05-022e10967d81 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:50.450192 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67687b3-c333-4933-8044-7124f55f3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:50.450837 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67687b3-c333-4933-8044-7124f55f3da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:50.451111 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67687b3-c333-4933-8044-7124f55f3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:50.451375 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67687b3-c333-4933-8044-7124f55f3da1 admin admin] PATH_INFO: `/v3/projects/119c156e97d94625ad19d671452d7727/users/e46c69248d374c3bb9b4f6a73143de33/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:50.453409 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67687b3-c333-4933-8044-7124f55f3da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=119c156e97d94625ad19d671452d7727, user_id=e46c69248d374c3bb9b4f6a73143de33, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:50.456146 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67687b3-c333-4933-8044-7124f55f3da1 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:50.467215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1314/2676] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:50 2022] PUT /identity/v3/projects/119c156e97d94625ad19d671452d7727/users/e46c69248d374c3bb9b4f6a73143de33/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:50.477692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794d512-1f5f-413f-a547-e7bc138c15de None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:50.478203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794d512-1f5f-413f-a547-e7bc138c15de None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:50.478306 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794d512-1f5f-413f-a547-e7bc138c15de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:50.504615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67687b3-c333-4933-8044-7124f55f3da1 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:50.515511 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:50.517697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-0794d512-1f5f-413f-a547-e7bc138c15de None None] MFA Rules not processed for user `e46c69248d374c3bb9b4f6a73143de33`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:50.518925 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0794d512-1f5f-413f-a547-e7bc138c15de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:50.579297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67687b3-c333-4933-8044-7124f55f3da1 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:50.584179 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ce78e-bb3e-445e-8a26-45fdefe95509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:50.584787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ce78e-bb3e-445e-8a26-45fdefe95509 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:50.585100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ce78e-bb3e-445e-8a26-45fdefe95509 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:50.585372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ce78e-bb3e-445e-8a26-45fdefe95509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:50.587129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ef6ce78e-bb3e-445e-8a26-45fdefe95509 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:50.599286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1363/2677] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:50.611484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794d512-1f5f-413f-a547-e7bc138c15de None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:50.612968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-0794d512-1f5f-413f-a547-e7bc138c15de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:50.626465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794d512-1f5f-413f-a547-e7bc138c15de None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:50.631045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d83fd-a709-48ae-a6a7-14d5e8b62fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:50.631661 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d83fd-a709-48ae-a6a7-14d5e8b62fba service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:50.632013 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d83fd-a709-48ae-a6a7-14d5e8b62fba service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:50.632332 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d83fd-a709-48ae-a6a7-14d5e8b62fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:50.643456 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d83fd-a709-48ae-a6a7-14d5e8b62fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:50.645440 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d83fd-a709-48ae-a6a7-14d5e8b62fba service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:50.652691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ce78e-bb3e-445e-8a26-45fdefe95509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:50.655034 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ce78e-bb3e-445e-8a26-45fdefe95509 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:50.662557 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1364/2678] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:50.666707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1315/2679] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:50 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:51.217462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d83fd-a709-48ae-a6a7-14d5e8b62fba service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:51.219260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d52d83fd-a709-48ae-a6a7-14d5e8b62fba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:51.299328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d83fd-a709-48ae-a6a7-14d5e8b62fba service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:51.301388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:51.302038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:51.302304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:51.302577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] PATH_INFO: `/v3/users/43ea575bacf944c5808c9f1d74988f07` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:51.303684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ea575bacf944c5808c9f1d74988f07)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:51.305707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:51.565086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:51.577340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ea575bacf944c5808c9f1d74988f07'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:51.702882 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ea575bacf944c5808c9f1d74988f07'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:51.719965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ea575bacf944c5808c9f1d74988f07'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:51.720735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ea575bacf944c5808c9f1d74988f07'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:51.727638 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ea575bacf944c5808c9f1d74988f07'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:51.734146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1365/2680] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:51 2022] DELETE /identity/v3/users/43ea575bacf944c5808c9f1d74988f07 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:51.738453 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ae0ab-c884-4019-afa1-ae4699c09db8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:51.738945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ae0ab-c884-4019-afa1-ae4699c09db8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:51.739126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ae0ab-c884-4019-afa1-ae4699c09db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:51.759843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-285ae0ab-c884-4019-afa1-ae4699c09db8 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:51.761845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-285ae0ab-c884-4019-afa1-ae4699c09db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:51.821982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1316/2681] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:51.832726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:51.834205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:51.919043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df617d26-807a-49f0-9949-e2c44d416dad admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:51.924469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192d534-17e0-44c9-8814-4074bd459abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:51.925309 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192d534-17e0-44c9-8814-4074bd459abf service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:51.925766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192d534-17e0-44c9-8814-4074bd459abf service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:51.926208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192d534-17e0-44c9-8814-4074bd459abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:51.939455 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192d534-17e0-44c9-8814-4074bd459abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:51.941236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192d534-17e0-44c9-8814-4074bd459abf service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:51.953766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1366/2682] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:52.265666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ae0ab-c884-4019-afa1-ae4699c09db8 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:52.267899 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-285ae0ab-c884-4019-afa1-ae4699c09db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:52.281352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ae0ab-c884-4019-afa1-ae4699c09db8 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:52.285945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:52.286577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:52.286860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:52.287129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] PATH_INFO: `/v3/users/33521fb407174bf6b60f65c1a5c25c63` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:52.288257 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33521fb407174bf6b60f65c1a5c25c63)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:52.290195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:52.457640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192d534-17e0-44c9-8814-4074bd459abf service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:52.459930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6192d534-17e0-44c9-8814-4074bd459abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:52.475093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192d534-17e0-44c9-8814-4074bd459abf service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:52.481026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:52.481689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:52.482116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:52.482236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] PATH_INFO: `/v3/projects/7a3ebe5cd4924ff19ddce156e9ac0a47` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:52.491282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a3ebe5cd4924ff19ddce156e9ac0a47)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:52.491282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:52.547965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33521fb407174bf6b60f65c1a5c25c63'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.548615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33521fb407174bf6b60f65c1a5c25c63'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.549344 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33521fb407174bf6b60f65c1a5c25c63'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.554312 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:52.557234 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33521fb407174bf6b60f65c1a5c25c63'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.570113 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-4e5a2835-620e-4036-b9e2-211beb446033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33521fb407174bf6b60f65c1a5c25c63'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.630591 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a3ebe5cd4924ff19ddce156e9ac0a47', 'request_id': 'req-9e345ba7-a0fd-4b77-afe5-804249e46bd4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.637534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a3ebe5cd4924ff19ddce156e9ac0a47', 'request_id': 'req-9e345ba7-a0fd-4b77-afe5-804249e46bd4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.699032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1317/2683] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:52 2022] DELETE /identity/v3/users/33521fb407174bf6b60f65c1a5c25c63 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:52.703250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badf5d7-df75-4d6f-93b2-45b7fd2928cd None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:52.703566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badf5d7-df75-4d6f-93b2-45b7fd2928cd None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:52.703845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badf5d7-df75-4d6f-93b2-45b7fd2928cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:52.723984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-1badf5d7-df75-4d6f-93b2-45b7fd2928cd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:52.725233 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1badf5d7-df75-4d6f-93b2-45b7fd2928cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:52.749939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a3ebe5cd4924ff19ddce156e9ac0a47', 'request_id': 'req-9e345ba7-a0fd-4b77-afe5-804249e46bd4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.764448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] The token cache is being invalidate because project 7a3ebe5cd4924ff19ddce156e9ac0a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:52.764736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.765429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:52.766893 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1367/2684] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:52 2022] DELETE /identity/v3/projects/7a3ebe5cd4924ff19ddce156e9ac0a47 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:52.771028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:52.772333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:52.793872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1318/2685] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:52.805018 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badf5d7-df75-4d6f-93b2-45b7fd2928cd None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:52.806442 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1badf5d7-df75-4d6f-93b2-45b7fd2928cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:52.817934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:52.842607 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e345ba7-a0fd-4b77-afe5-804249e46bd4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:52.848852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:52.849493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:52.849770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:52.850040 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] PATH_INFO: `/v3/users/42eed51989364b6083fa1c47c4f33fb3` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:52.851171 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42eed51989364b6083fa1c47c4f33fb3)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:52.853112 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:52.868819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1badf5d7-df75-4d6f-93b2-45b7fd2928cd None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:52.872836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407de647-2a3a-4584-9504-030d88b18715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:52.873610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407de647-2a3a-4584-9504-030d88b18715 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:52.875520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407de647-2a3a-4584-9504-030d88b18715 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:52.875520 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407de647-2a3a-4584-9504-030d88b18715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:52.884186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407de647-2a3a-4584-9504-030d88b18715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:52.886804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407de647-2a3a-4584-9504-030d88b18715 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:52.897534 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1319/2686] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:53.082352 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42eed51989364b6083fa1c47c4f33fb3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.153088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42eed51989364b6083fa1c47c4f33fb3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.167868 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42eed51989364b6083fa1c47c4f33fb3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.169032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42eed51989364b6083fa1c47c4f33fb3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.177261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42eed51989364b6083fa1c47c4f33fb3'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.184293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1368/2687] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:52 2022] DELETE /identity/v3/users/42eed51989364b6083fa1c47c4f33fb3 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:53.348831 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407de647-2a3a-4584-9504-030d88b18715 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:53.350946 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-407de647-2a3a-4584-9504-030d88b18715 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:53.378429 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407de647-2a3a-4584-9504-030d88b18715 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:53.439535 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:53.440175 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:53.440476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:53.440746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] PATH_INFO: `/v3/projects/efe2ef9f238548b7a12d56888fc8ceaa` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:53.442115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe2ef9f238548b7a12d56888fc8ceaa)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:53.444084 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:53.462266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:53.565386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe2ef9f238548b7a12d56888fc8ceaa', 'request_id': 'req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.574363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe2ef9f238548b7a12d56888fc8ceaa', 'request_id': 'req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.680718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe2ef9f238548b7a12d56888fc8ceaa', 'request_id': 'req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.693684 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] The token cache is being invalidate because project efe2ef9f238548b7a12d56888fc8ceaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:53.693985 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.694732 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:53.696291 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1320/2688] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:53 2022] DELETE /identity/v3/projects/efe2ef9f238548b7a12d56888fc8ceaa => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:53.706068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:53.707764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:53.715953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:53.718238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:53.724265 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188bbd30-fad9-433a-8269-6ef7de233110 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:53.789880 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398ff8b-65c1-444a-bea1-a5a5622c9b9a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:53.794911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:53.795552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:53.795832 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:53.796104 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] PATH_INFO: `/v3/projects/108762798fb648b0a3b146feaa3667eb` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:53.796617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:53.797342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:53.797471 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=108762798fb648b0a3b146feaa3667eb)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:53.797880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:53.798284 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] PATH_INFO: `/v3/users/e46c69248d374c3bb9b4f6a73143de33` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:53.799457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46c69248d374c3bb9b4f6a73143de33)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:53.799693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:53.801390 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:54.026838 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108762798fb648b0a3b146feaa3667eb', 'request_id': 'req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.027075 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46c69248d374c3bb9b4f6a73143de33'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.033310 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108762798fb648b0a3b146feaa3667eb', 'request_id': 'req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.115429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46c69248d374c3bb9b4f6a73143de33'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.121603 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:54.130189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46c69248d374c3bb9b4f6a73143de33'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.130899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46c69248d374c3bb9b4f6a73143de33'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.139895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46c69248d374c3bb9b4f6a73143de33'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.146001 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1369/2689] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:53 2022] DELETE /identity/v3/users/e46c69248d374c3bb9b4f6a73143de33 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:54.258020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108762798fb648b0a3b146feaa3667eb', 'request_id': 'req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.270715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] The token cache is being invalidate because project 108762798fb648b0a3b146feaa3667eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:54.273614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.273614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.273614 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1321/2690] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:53 2022] DELETE /identity/v3/projects/108762798fb648b0a3b146feaa3667eb => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:54.277234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:54.278564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:54.347198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb849b-4d6a-431f-910a-eb22561de450 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:54.351987 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:54.352639 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:54.352955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:54.353231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] PATH_INFO: `/v3/users/4544945742084a448dc0fc6df46573d2` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:54.354982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4544945742084a448dc0fc6df46573d2)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:54.356954 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:54.613351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4544945742084a448dc0fc6df46573d2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.658564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:54.660025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:54.685559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4544945742084a448dc0fc6df46573d2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.699164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4544945742084a448dc0fc6df46573d2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.699975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4544945742084a448dc0fc6df46573d2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.700813 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:54.707393 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-fbf565a7-7839-4ddb-ba52-7b084e07ed26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4544945742084a448dc0fc6df46573d2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.715389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1370/2691] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:54 2022] DELETE /identity/v3/users/4544945742084a448dc0fc6df46573d2 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:54.733783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcc6ba-0cf0-4638-a9e1-0cdf9bce92fc admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:54.791399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:54.792815 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:54.793218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:54.793493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] PATH_INFO: `/v3/projects/119c156e97d94625ad19d671452d7727` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:54.794658 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=119c156e97d94625ad19d671452d7727)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:54.796615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:54.923164 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '119c156e97d94625ad19d671452d7727', 'request_id': 'req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:54.928964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '119c156e97d94625ad19d671452d7727', 'request_id': 'req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:55.023337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '119c156e97d94625ad19d671452d7727', 'request_id': 'req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:55.036745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] The token cache is being invalidate because project 119c156e97d94625ad19d671452d7727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:55.037067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:55.040661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-d7cb8d8f-59e7-454e-bf0b-e993f0caf191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:55.040661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1322/2692] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:54 2022] DELETE /identity/v3/projects/119c156e97d94625ad19d671452d7727 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:55.045567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307fc0-c362-4852-aab7-b28148b7a239 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.045859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307fc0-c362-4852-aab7-b28148b7a239 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.046136 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307fc0-c362-4852-aab7-b28148b7a239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.066832 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-2c307fc0-c362-4852-aab7-b28148b7a239 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:55.068043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2c307fc0-c362-4852-aab7-b28148b7a239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:55.128569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1371/2693] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:55.137183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4b07e-80f2-48e9-8d98-99b1a8b73fdd None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.137460 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4b07e-80f2-48e9-8d98-99b1a8b73fdd None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.137736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4b07e-80f2-48e9-8d98-99b1a8b73fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.159115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-6ff4b07e-80f2-48e9-8d98-99b1a8b73fdd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:55.159874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6ff4b07e-80f2-48e9-8d98-99b1a8b73fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:55.169125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1323/2694] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:55.170914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c307fc0-c362-4852-aab7-b28148b7a239 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.172350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-2c307fc0-c362-4852-aab7-b28148b7a239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:55.172933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4b07e-80f2-48e9-8d98-99b1a8b73fdd None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.184539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c307fc0-c362-4852-aab7-b28148b7a239 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.189724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.192363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.192668 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.193017 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] PATH_INFO: `/v3/projects/aa4cf87e2db44780b5ad9885c3922020` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.194100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4b07e-80f2-48e9-8d98-99b1a8b73fdd None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.194280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4cf87e2db44780b5ad9885c3922020)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.196546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.202313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74640b96-0abc-432c-9030-4e6b7d49902a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.202313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74640b96-0abc-432c-9030-4e6b7d49902a admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.202313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74640b96-0abc-432c-9030-4e6b7d49902a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.202313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74640b96-0abc-432c-9030-4e6b7d49902a admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.202705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74640b96-0abc-432c-9030-4e6b7d49902a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:55.202705 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74640b96-0abc-432c-9030-4e6b7d49902a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.204362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74640b96-0abc-432c-9030-4e6b7d49902a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.217624 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1324/2695] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:46:55.222137 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74640b96-0abc-432c-9030-4e6b7d49902a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.225014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:55.232904 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74640b96-0abc-432c-9030-4e6b7d49902a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.237618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb946e-34db-4368-86c9-4d304aade397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.238208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb946e-34db-4368-86c9-4d304aade397 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.238490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb946e-34db-4368-86c9-4d304aade397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.238764 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb946e-34db-4368-86c9-4d304aade397 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.239384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb946e-34db-4368-86c9-4d304aade397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.241317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb946e-34db-4368-86c9-4d304aade397 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.345372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1325/2696] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:55 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:55.349488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb946e-34db-4368-86c9-4d304aade397 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.359000 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4cf87e2db44780b5ad9885c3922020', 'request_id': 'req-d7ed9925-2549-4093-8e9a-f3d5e9472952'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:55.365312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4cf87e2db44780b5ad9885c3922020', 'request_id': 'req-d7ed9925-2549-4093-8e9a-f3d5e9472952'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:55.414588 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb946e-34db-4368-86c9-4d304aade397 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.420148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900ae22-f087-47fa-83da-5814cca06779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.420860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900ae22-f087-47fa-83da-5814cca06779 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.421148 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900ae22-f087-47fa-83da-5814cca06779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.421418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900ae22-f087-47fa-83da-5814cca06779 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.422102 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900ae22-f087-47fa-83da-5814cca06779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.424082 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900ae22-f087-47fa-83da-5814cca06779 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.463532 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4cf87e2db44780b5ad9885c3922020', 'request_id': 'req-d7ed9925-2549-4093-8e9a-f3d5e9472952'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:55.475370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] The token cache is being invalidate because project aa4cf87e2db44780b5ad9885c3922020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:55.475659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:55.476310 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-d7ed9925-2549-4093-8e9a-f3d5e9472952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:55.477837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1372/2697] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:55 2022] DELETE /identity/v3/projects/aa4cf87e2db44780b5ad9885c3922020 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:55.484341 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a008fe-fdcd-411a-93f4-c00223c6a8a2 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.484647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a008fe-fdcd-411a-93f4-c00223c6a8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.484949 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a008fe-fdcd-411a-93f4-c00223c6a8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.497057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:55.505491 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f6a008fe-fdcd-411a-93f4-c00223c6a8a2 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:55.506699 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f6a008fe-fdcd-411a-93f4-c00223c6a8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:55.515858 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1373/2698] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:55.524605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6c54d-7972-41ba-99b4-478cb916016f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.524930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6c54d-7972-41ba-99b4-478cb916016f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.525198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf6c54d-7972-41ba-99b4-478cb916016f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.544823 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-baf6c54d-7972-41ba-99b4-478cb916016f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:55.546052 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-baf6c54d-7972-41ba-99b4-478cb916016f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:55.555563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1374/2699] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:55.560791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6c54d-7972-41ba-99b4-478cb916016f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.564681 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1326/2700] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:55 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:55.565497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900ae22-f087-47fa-83da-5814cca06779 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.567419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d900ae22-f087-47fa-83da-5814cca06779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:55.570891 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf6c54d-7972-41ba-99b4-478cb916016f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.579934 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2ee0e-bc8b-4ad2-9916-c3e6220f70ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.580342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900ae22-f087-47fa-83da-5814cca06779 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.580535 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2ee0e-bc8b-4ad2-9916-c3e6220f70ce admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.580833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2ee0e-bc8b-4ad2-9916-c3e6220f70ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.581110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2ee0e-bc8b-4ad2-9916-c3e6220f70ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.581774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2ee0e-bc8b-4ad2-9916-c3e6220f70ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:55.582225 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2ee0e-bc8b-4ad2-9916-c3e6220f70ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.583804 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2ee0e-bc8b-4ad2-9916-c3e6220f70ce admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.585400 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79662d18-14cf-48f8-bb4c-e46bcdb07d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.585938 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79662d18-14cf-48f8-bb4c-e46bcdb07d52 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.586218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79662d18-14cf-48f8-bb4c-e46bcdb07d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.586485 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79662d18-14cf-48f8-bb4c-e46bcdb07d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.587109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79662d18-14cf-48f8-bb4c-e46bcdb07d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.588775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79662d18-14cf-48f8-bb4c-e46bcdb07d52 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.597088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1375/2701] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:46:55.602182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1327/2702] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:55.611431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2ee0e-bc8b-4ad2-9916-c3e6220f70ce admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.613710 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79662d18-14cf-48f8-bb4c-e46bcdb07d52 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.622382 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2ee0e-bc8b-4ad2-9916-c3e6220f70ce admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.625079 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79662d18-14cf-48f8-bb4c-e46bcdb07d52 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.627533 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b50d74-3d7e-45fd-a8d0-00046dda7a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.628135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50d74-3d7e-45fd-a8d0-00046dda7a79 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.628410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50d74-3d7e-45fd-a8d0-00046dda7a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.628677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50d74-3d7e-45fd-a8d0-00046dda7a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.629340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b50d74-3d7e-45fd-a8d0-00046dda7a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.630379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc654fc-6ca9-42cd-afa7-c8cfb82ede8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.630928 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b50d74-3d7e-45fd-a8d0-00046dda7a79 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.631054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc654fc-6ca9-42cd-afa7-c8cfb82ede8f admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.631428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc654fc-6ca9-42cd-afa7-c8cfb82ede8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.631576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc654fc-6ca9-42cd-afa7-c8cfb82ede8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.632198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc654fc-6ca9-42cd-afa7-c8cfb82ede8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.634116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc654fc-6ca9-42cd-afa7-c8cfb82ede8f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.642540 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1376/2703] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:55.646806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b50d74-3d7e-45fd-a8d0-00046dda7a79 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.663811 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b50d74-3d7e-45fd-a8d0-00046dda7a79 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.671373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0aa27b-5f79-4bdc-b8c6-bfec9aead657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.671373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0aa27b-5f79-4bdc-b8c6-bfec9aead657 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.671373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0aa27b-5f79-4bdc-b8c6-bfec9aead657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.671373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0aa27b-5f79-4bdc-b8c6-bfec9aead657 admin admin] PATH_INFO: `/v3/projects/a40fb9a256b4483a85ea730c48621070/users/4bda959c0c0e406d9a02fbeb53fb2db2/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.694920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0aa27b-5f79-4bdc-b8c6-bfec9aead657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a40fb9a256b4483a85ea730c48621070, user_id=4bda959c0c0e406d9a02fbeb53fb2db2, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.698778 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0aa27b-5f79-4bdc-b8c6-bfec9aead657 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.757235 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1328/2704] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:55 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:46:55.762365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc654fc-6ca9-42cd-afa7-c8cfb82ede8f admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.786529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1377/2705] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:55 2022] PUT /identity/v3/projects/a40fb9a256b4483a85ea730c48621070/users/4bda959c0c0e406d9a02fbeb53fb2db2/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:55.800078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3dbfc-3c94-4291-906d-6da740004754 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.800361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3dbfc-3c94-4291-906d-6da740004754 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.800632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3dbfc-3c94-4291-906d-6da740004754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.828995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc654fc-6ca9-42cd-afa7-c8cfb82ede8f admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.833517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b5d3dbfc-3c94-4291-906d-6da740004754 None None] MFA Rules not processed for user `4bda959c0c0e406d9a02fbeb53fb2db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:55.833711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b5d3dbfc-3c94-4291-906d-6da740004754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:55.893470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcdbde-01a6-42af-9c56-d39df78b0aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.894644 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcdbde-01a6-42af-9c56-d39df78b0aca admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.895387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcdbde-01a6-42af-9c56-d39df78b0aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.895834 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcdbde-01a6-42af-9c56-d39df78b0aca admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:55.896477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcdbde-01a6-42af-9c56-d39df78b0aca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:55.898604 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcdbde-01a6-42af-9c56-d39df78b0aca admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:55.904334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1378/2706] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:46:55.916103 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3dbfc-3c94-4291-906d-6da740004754 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:55.918282 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b5d3dbfc-3c94-4291-906d-6da740004754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:55.928288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:55.990387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3dbfc-3c94-4291-906d-6da740004754 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:55.996286 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea1d49-2328-47ab-abd8-ad327a03ecdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:55.996942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea1d49-2328-47ab-abd8-ad327a03ecdd service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:55.997308 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea1d49-2328-47ab-abd8-ad327a03ecdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:55.997586 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea1d49-2328-47ab-abd8-ad327a03ecdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:56.010490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea1d49-2328-47ab-abd8-ad327a03ecdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:56.011186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1329/2707] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:46:55 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:46:56.012957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea1d49-2328-47ab-abd8-ad327a03ecdd service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:56.013094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcdbde-01a6-42af-9c56-d39df78b0aca admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:56.023503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcdbde-01a6-42af-9c56-d39df78b0aca admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:56.025659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1379/2708] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:46:56.028539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267769-d216-457a-9c57-07bc9a360ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:56.029182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267769-d216-457a-9c57-07bc9a360ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:56.029484 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267769-d216-457a-9c57-07bc9a360ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:56.029723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267769-d216-457a-9c57-07bc9a360ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:56.030334 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267769-d216-457a-9c57-07bc9a360ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:56.031956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267769-d216-457a-9c57-07bc9a360ea6 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:56.044820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1330/2709] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:46:56.052786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea1d49-2328-47ab-abd8-ad327a03ecdd service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:56.064618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea1d49-2328-47ab-abd8-ad327a03ecdd service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:56.069196 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680db25-fcf2-4a41-8619-82bd8348276c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:56.069797 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680db25-fcf2-4a41-8619-82bd8348276c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:56.070069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680db25-fcf2-4a41-8619-82bd8348276c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:56.070344 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680db25-fcf2-4a41-8619-82bd8348276c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:56.071201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680db25-fcf2-4a41-8619-82bd8348276c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:56.072581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680db25-fcf2-4a41-8619-82bd8348276c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:56.083356 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1380/2710] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:46:56.087670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267769-d216-457a-9c57-07bc9a360ea6 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:56.098691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267769-d216-457a-9c57-07bc9a360ea6 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:56.103527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280bd22-f4a7-4e08-ba8f-e24691faaeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:56.104177 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280bd22-f4a7-4e08-ba8f-e24691faaeef admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:56.105051 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280bd22-f4a7-4e08-ba8f-e24691faaeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:56.105374 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280bd22-f4a7-4e08-ba8f-e24691faaeef admin admin] PATH_INFO: `/v3/projects/af412f93b9ad4198b7fbfc3bf5863318/users/cdc5e36c0c7745278eeecacc569c03dd/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:56.114057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:56.122847 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280bd22-f4a7-4e08-ba8f-e24691faaeef admin admin] RBAC: Authorizing `identity:create_grant(project_id=af412f93b9ad4198b7fbfc3bf5863318, user_id=cdc5e36c0c7745278eeecacc569c03dd, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:56.127916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280bd22-f4a7-4e08-ba8f-e24691faaeef admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:56.219260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1331/2711] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:46:56 2022] PUT /identity/v3/projects/af412f93b9ad4198b7fbfc3bf5863318/users/cdc5e36c0c7745278eeecacc569c03dd/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:46:56.230958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dc8d6-e901-4bd2-8d81-df0cc9b698c7 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:56.231243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dc8d6-e901-4bd2-8d81-df0cc9b698c7 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:56.231516 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dc8d6-e901-4bd2-8d81-df0cc9b698c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:56.260740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f93dc8d6-e901-4bd2-8d81-df0cc9b698c7 None None] MFA Rules not processed for user `cdc5e36c0c7745278eeecacc569c03dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:56.261972 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f93dc8d6-e901-4bd2-8d81-df0cc9b698c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:56.331234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1381/2712] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:56.345218 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280bd22-f4a7-4e08-ba8f-e24691faaeef admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:56.347077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d280bd22-f4a7-4e08-ba8f-e24691faaeef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:56.408513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280bd22-f4a7-4e08-ba8f-e24691faaeef admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:56.412387 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0261daf-e505-4dcb-91cd-b5f26977810b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:56.413009 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261daf-e505-4dcb-91cd-b5f26977810b service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:56.413281 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261daf-e505-4dcb-91cd-b5f26977810b service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:56.413544 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0261daf-e505-4dcb-91cd-b5f26977810b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:56.422756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0261daf-e505-4dcb-91cd-b5f26977810b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:56.429753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0261daf-e505-4dcb-91cd-b5f26977810b service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:56.433089 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1332/2713] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:57.086664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dc8d6-e901-4bd2-8d81-df0cc9b698c7 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:57.088454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f93dc8d6-e901-4bd2-8d81-df0cc9b698c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:57.171483 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dc8d6-e901-4bd2-8d81-df0cc9b698c7 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:57.176751 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:57.177576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:57.177901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:57.178328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] PATH_INFO: `/v3/users/d7fbb661dc8c4b66a4916ec380582267` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:57.179584 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7fbb661dc8c4b66a4916ec380582267)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:57.182032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:57.413119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:57.427173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7fbb661dc8c4b66a4916ec380582267'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:57.547054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7fbb661dc8c4b66a4916ec380582267'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:57.563567 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7fbb661dc8c4b66a4916ec380582267'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:57.564207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7fbb661dc8c4b66a4916ec380582267'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:57.571231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7fbb661dc8c4b66a4916ec380582267'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:57.577525 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1382/2714] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:57 2022] DELETE /identity/v3/users/d7fbb661dc8c4b66a4916ec380582267 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:57.583314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676bf8f-7021-46f2-b30f-c8b475a3c3dc None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:57.583835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676bf8f-7021-46f2-b30f-c8b475a3c3dc None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:57.583835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676bf8f-7021-46f2-b30f-c8b475a3c3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:57.603690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-2676bf8f-7021-46f2-b30f-c8b475a3c3dc None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:57.605994 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2676bf8f-7021-46f2-b30f-c8b475a3c3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:57.676124 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1333/2715] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:57.686153 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:57.687601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:57.747110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17a089-7108-41f7-9f98-9d735fc6d33a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:57.750649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733a975-ea10-4018-a973-197f44c03455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:57.751511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733a975-ea10-4018-a973-197f44c03455 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:57.751846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733a975-ea10-4018-a973-197f44c03455 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:57.752058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733a975-ea10-4018-a973-197f44c03455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:57.761275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733a975-ea10-4018-a973-197f44c03455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:57.762693 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733a975-ea10-4018-a973-197f44c03455 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:57.772168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1383/2716] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:46:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:46:58.242445 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676bf8f-7021-46f2-b30f-c8b475a3c3dc None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:58.243701 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2676bf8f-7021-46f2-b30f-c8b475a3c3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:58.257552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676bf8f-7021-46f2-b30f-c8b475a3c3dc None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:58.264283 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:58.265414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:58.265807 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:58.266398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] PATH_INFO: `/v3/projects/2ec1ed87aa724146976ca73ef05f48a3` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:58.267736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec1ed87aa724146976ca73ef05f48a3)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:58.269719 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:58.392040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec1ed87aa724146976ca73ef05f48a3', 'request_id': 'req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.399640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec1ed87aa724146976ca73ef05f48a3', 'request_id': 'req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.512928 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec1ed87aa724146976ca73ef05f48a3', 'request_id': 'req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.525300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] The token cache is being invalidate because project 2ec1ed87aa724146976ca73ef05f48a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:58.525617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.526289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.527824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1334/2717] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:58 2022] DELETE /identity/v3/projects/2ec1ed87aa724146976ca73ef05f48a3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:58.531844 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733a975-ea10-4018-a973-197f44c03455 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:58.541800 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733a975-ea10-4018-a973-197f44c03455 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:58.606143 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:58.606764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:58.607043 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:58.607448 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] PATH_INFO: `/v3/users/6bbaa71b038e4256aef0ad0d453d08b1` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:58.608563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bbaa71b038e4256aef0ad0d453d08b1)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:58.610479 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:58.829080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bbaa71b038e4256aef0ad0d453d08b1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.832713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:58.943454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bbaa71b038e4256aef0ad0d453d08b1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.958573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bbaa71b038e4256aef0ad0d453d08b1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.959290 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bbaa71b038e4256aef0ad0d453d08b1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.967085 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-40ad6f3e-6f34-46be-89c8-c0ccbc3cf6a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bbaa71b038e4256aef0ad0d453d08b1'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:58.976705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1384/2718] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:46:58 2022] DELETE /identity/v3/users/6bbaa71b038e4256aef0ad0d453d08b1 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:59.426337 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:59.427841 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:59.451529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f6323-6320-4452-bbe7-2a3f97f9f43e admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:59.465094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:46:59.514280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:59.514921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:59.515303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:59.515584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] PATH_INFO: `/v3/projects/fc498272f2194598b18758ed04694331` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:59.516699 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc498272f2194598b18758ed04694331)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:59.518627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:59.616597 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc498272f2194598b18758ed04694331', 'request_id': 'req-55e041b0-d01e-4f39-b260-1258e7a32d0a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:59.623213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc498272f2194598b18758ed04694331', 'request_id': 'req-55e041b0-d01e-4f39-b260-1258e7a32d0a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:59.763090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc498272f2194598b18758ed04694331', 'request_id': 'req-55e041b0-d01e-4f39-b260-1258e7a32d0a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:59.781703 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] The token cache is being invalidate because project fc498272f2194598b18758ed04694331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:46:59.782108 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:59.782874 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-55e041b0-d01e-4f39-b260-1258e7a32d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:46:59.784750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1335/2719] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:46:59 2022] DELETE /identity/v3/projects/fc498272f2194598b18758ed04694331 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:46:59.791631 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf246df6-23bb-48c0-a5fd-ddfefaa8d5ab None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:59.792367 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf246df6-23bb-48c0-a5fd-ddfefaa8d5ab None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:59.792575 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf246df6-23bb-48c0-a5fd-ddfefaa8d5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:59.815877 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cf246df6-23bb-48c0-a5fd-ddfefaa8d5ab None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:59.817115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cf246df6-23bb-48c0-a5fd-ddfefaa8d5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:59.880425 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1385/2720] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:59.889479 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22c34a-f421-4ae4-a5a9-e05ef4c50247 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:59.890032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22c34a-f421-4ae4-a5a9-e05ef4c50247 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:59.890512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22c34a-f421-4ae4-a5a9-e05ef4c50247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:59.910708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-cb22c34a-f421-4ae4-a5a9-e05ef4c50247 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:46:59.911930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cb22c34a-f421-4ae4-a5a9-e05ef4c50247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:46:59.920637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1336/2721] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:46:59.924978 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf246df6-23bb-48c0-a5fd-ddfefaa8d5ab None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:59.936115 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf246df6-23bb-48c0-a5fd-ddfefaa8d5ab None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:59.940617 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07159e87-b37e-4b8a-9c14-b5fdd0013b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:59.941293 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07159e87-b37e-4b8a-9c14-b5fdd0013b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:59.941568 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07159e87-b37e-4b8a-9c14-b5fdd0013b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:59.941835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07159e87-b37e-4b8a-9c14-b5fdd0013b21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:59.942469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07159e87-b37e-4b8a-9c14-b5fdd0013b21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:46:59.942760 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07159e87-b37e-4b8a-9c14-b5fdd0013b21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:59.944331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07159e87-b37e-4b8a-9c14-b5fdd0013b21 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:46:59.957164 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1386/2722] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:46:59.961661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22c34a-f421-4ae4-a5a9-e05ef4c50247 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:46:59.971569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22c34a-f421-4ae4-a5a9-e05ef4c50247 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:46:59.976470 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb238c2-0da2-49d8-b50d-9e5e09ba37da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:46:59.977026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb238c2-0da2-49d8-b50d-9e5e09ba37da admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:46:59.977305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb238c2-0da2-49d8-b50d-9e5e09ba37da admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:46:59.977578 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb238c2-0da2-49d8-b50d-9e5e09ba37da admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:46:59.978261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb238c2-0da2-49d8-b50d-9e5e09ba37da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:46:59.980891 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb238c2-0da2-49d8-b50d-9e5e09ba37da admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:00.120579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1337/2723] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:46:59 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:47:00.125256 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07159e87-b37e-4b8a-9c14-b5fdd0013b21 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:00.137077 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07159e87-b37e-4b8a-9c14-b5fdd0013b21 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:00.200588 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edeff0-2a6b-4424-b4b0-452dc97d753b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:00.201235 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edeff0-2a6b-4424-b4b0-452dc97d753b admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:00.201517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edeff0-2a6b-4424-b4b0-452dc97d753b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:00.201790 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edeff0-2a6b-4424-b4b0-452dc97d753b admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:00.202420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edeff0-2a6b-4424-b4b0-452dc97d753b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:00.204347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edeff0-2a6b-4424-b4b0-452dc97d753b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:00.308123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1387/2724] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:47:00 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:47:00.312158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb238c2-0da2-49d8-b50d-9e5e09ba37da admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:00.322693 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb238c2-0da2-49d8-b50d-9e5e09ba37da admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:00.327583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e319b-4097-4eed-97c4-eb28aaacbd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:00.328186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e319b-4097-4eed-97c4-eb28aaacbd32 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:00.328462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e319b-4097-4eed-97c4-eb28aaacbd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:00.328722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e319b-4097-4eed-97c4-eb28aaacbd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:00.329469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e319b-4097-4eed-97c4-eb28aaacbd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:00.331049 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e319b-4097-4eed-97c4-eb28aaacbd32 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:00.341893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1338/2725] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:47:00.345954 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edeff0-2a6b-4424-b4b0-452dc97d753b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:00.356905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edeff0-2a6b-4424-b4b0-452dc97d753b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:00.361764 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7b6d-5b58-4d26-88b1-a5062c27d861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:00.362372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7b6d-5b58-4d26-88b1-a5062c27d861 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:00.362648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7b6d-5b58-4d26-88b1-a5062c27d861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:00.362914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7b6d-5b58-4d26-88b1-a5062c27d861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:00.363550 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a7b6d-5b58-4d26-88b1-a5062c27d861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:00.365257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a7b6d-5b58-4d26-88b1-a5062c27d861 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:00.377510 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1388/2726] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:47:00.397865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e319b-4097-4eed-97c4-eb28aaacbd32 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:00.408136 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e319b-4097-4eed-97c4-eb28aaacbd32 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:00.413270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af9c5d-eca0-45d8-bcdd-b18407870924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:00.413929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af9c5d-eca0-45d8-bcdd-b18407870924 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:00.414202 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af9c5d-eca0-45d8-bcdd-b18407870924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:00.414467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af9c5d-eca0-45d8-bcdd-b18407870924 admin admin] PATH_INFO: `/v3/projects/edb50adb174e4348aea40289a23b4c18/users/ca82ac2473d242e0b4244fec89cbbec8/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:00.431390 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af9c5d-eca0-45d8-bcdd-b18407870924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb50adb174e4348aea40289a23b4c18, user_id=ca82ac2473d242e0b4244fec89cbbec8, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:00.434820 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af9c5d-eca0-45d8-bcdd-b18407870924 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:00.507757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1339/2727] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:00 2022] PUT /identity/v3/projects/edb50adb174e4348aea40289a23b4c18/users/ca82ac2473d242e0b4244fec89cbbec8/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:00.519700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a0ffd-c504-4ac3-ad76-0ec0b6730572 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:00.519980 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a0ffd-c504-4ac3-ad76-0ec0b6730572 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:00.520267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a0ffd-c504-4ac3-ad76-0ec0b6730572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:00.549702 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-a75a0ffd-c504-4ac3-ad76-0ec0b6730572 None None] MFA Rules not processed for user `ca82ac2473d242e0b4244fec89cbbec8`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:00.550938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a75a0ffd-c504-4ac3-ad76-0ec0b6730572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:00.575689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:00.617232 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1389/2728] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:00.622282 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af9c5d-eca0-45d8-bcdd-b18407870924 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:00.670256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:00.690333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af9c5d-eca0-45d8-bcdd-b18407870924 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:00.695107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b5378-7490-4277-89bd-7d5a7c691818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:00.695714 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b5378-7490-4277-89bd-7d5a7c691818 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:00.695993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b5378-7490-4277-89bd-7d5a7c691818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:00.696265 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b5378-7490-4277-89bd-7d5a7c691818 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:00.697109 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b5378-7490-4277-89bd-7d5a7c691818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:00.700200 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b5378-7490-4277-89bd-7d5a7c691818 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:00.848734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1340/2729] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:47:00 2022] POST /identity/v3/projects => generated 381 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:47:00.852909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a0ffd-c504-4ac3-ad76-0ec0b6730572 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:00.864642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a0ffd-c504-4ac3-ad76-0ec0b6730572 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:00.923888 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dcb2c-f365-4a7a-9d45-0cf2c1043a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:00.924505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dcb2c-f365-4a7a-9d45-0cf2c1043a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:00.924783 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dcb2c-f365-4a7a-9d45-0cf2c1043a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:00.925084 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dcb2c-f365-4a7a-9d45-0cf2c1043a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:00.925715 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dcb2c-f365-4a7a-9d45-0cf2c1043a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:00.927640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dcb2c-f365-4a7a-9d45-0cf2c1043a24 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:01.058400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1390/2730] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:47:00 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:47:01.062618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b5378-7490-4277-89bd-7d5a7c691818 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:01.073367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b5378-7490-4277-89bd-7d5a7c691818 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:01.078661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0baa66-b224-4e15-9194-872e80c6d111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:01.079296 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0baa66-b224-4e15-9194-872e80c6d111 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:01.079633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0baa66-b224-4e15-9194-872e80c6d111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:01.079935 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0baa66-b224-4e15-9194-872e80c6d111 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:01.080563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0baa66-b224-4e15-9194-872e80c6d111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:01.084161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0baa66-b224-4e15-9194-872e80c6d111 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:01.093763 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1341/2731] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:47:01.097958 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dcb2c-f365-4a7a-9d45-0cf2c1043a24 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:01.108458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dcb2c-f365-4a7a-9d45-0cf2c1043a24 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:01.113573 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc5bdc-4b6d-4be5-989f-8c55239a9177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:01.114271 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc5bdc-4b6d-4be5-989f-8c55239a9177 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:01.114627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc5bdc-4b6d-4be5-989f-8c55239a9177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:01.114968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc5bdc-4b6d-4be5-989f-8c55239a9177 admin admin] PATH_INFO: `/v3/projects/bfc60d7559894255998f9c431aa1e61b/users/a3f1ada700fe4c77a69dc01761668a54/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:01.131855 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc5bdc-4b6d-4be5-989f-8c55239a9177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc60d7559894255998f9c431aa1e61b, user_id=a3f1ada700fe4c77a69dc01761668a54, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:01.134849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc5bdc-4b6d-4be5-989f-8c55239a9177 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:01.205292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1391/2732] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:01 2022] PUT /identity/v3/projects/bfc60d7559894255998f9c431aa1e61b/users/a3f1ada700fe4c77a69dc01761668a54/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:01.209351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0baa66-b224-4e15-9194-872e80c6d111 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:01.220674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0baa66-b224-4e15-9194-872e80c6d111 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:01.279351 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfc75e-a615-42b9-82d7-eacbf13f803f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:01.279963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfc75e-a615-42b9-82d7-eacbf13f803f admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:01.280241 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfc75e-a615-42b9-82d7-eacbf13f803f admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:01.280512 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfc75e-a615-42b9-82d7-eacbf13f803f admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:01.281181 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfc75e-a615-42b9-82d7-eacbf13f803f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:01.282772 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfc75e-a615-42b9-82d7-eacbf13f803f admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:01.293454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1342/2733] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:47:01.297723 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc5bdc-4b6d-4be5-989f-8c55239a9177 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:01.309019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc5bdc-4b6d-4be5-989f-8c55239a9177 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:01.313663 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716f789-529c-45bb-822e-a6a2bb427ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:01.314325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716f789-529c-45bb-822e-a6a2bb427ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:01.314601 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716f789-529c-45bb-822e-a6a2bb427ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:01.314872 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716f789-529c-45bb-822e-a6a2bb427ce5 admin admin] PATH_INFO: `/v3/projects/bfc60d7559894255998f9c431aa1e61b/users/a3f1ada700fe4c77a69dc01761668a54/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:01.316782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716f789-529c-45bb-822e-a6a2bb427ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc60d7559894255998f9c431aa1e61b, user_id=a3f1ada700fe4c77a69dc01761668a54, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:01.319495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716f789-529c-45bb-822e-a6a2bb427ce5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:01.331234 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1392/2734] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:01 2022] PUT /identity/v3/projects/bfc60d7559894255998f9c431aa1e61b/users/a3f1ada700fe4c77a69dc01761668a54/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:01.355692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f553-4843-474a-a5ea-4455656ade0b None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:01.355692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f553-4843-474a-a5ea-4455656ade0b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:01.355692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f553-4843-474a-a5ea-4455656ade0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:01.372158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-7619f553-4843-474a-a5ea-4455656ade0b None None] MFA Rules not processed for user `a3f1ada700fe4c77a69dc01761668a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:01.373435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7619f553-4843-474a-a5ea-4455656ade0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:01.445406 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1343/2735] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:01.456944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716f789-529c-45bb-822e-a6a2bb427ce5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:01.458735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-a716f789-529c-45bb-822e-a6a2bb427ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:01.521865 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716f789-529c-45bb-822e-a6a2bb427ce5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:01.525501 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f73fe3-00ef-4697-87a7-3a914ce5d132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:01.526291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f73fe3-00ef-4697-87a7-3a914ce5d132 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:01.526682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f73fe3-00ef-4697-87a7-3a914ce5d132 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:01.526959 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f73fe3-00ef-4697-87a7-3a914ce5d132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:01.537259 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f73fe3-00ef-4697-87a7-3a914ce5d132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:01.538750 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f73fe3-00ef-4697-87a7-3a914ce5d132 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:01.549047 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1393/2736] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:01.658640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f553-4843-474a-a5ea-4455656ade0b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:01.669605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f553-4843-474a-a5ea-4455656ade0b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:01.673169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89d20a-ff71-4926-9c8e-4e2a38076385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:01.673783 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89d20a-ff71-4926-9c8e-4e2a38076385 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:01.674058 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89d20a-ff71-4926-9c8e-4e2a38076385 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:01.674324 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89d20a-ff71-4926-9c8e-4e2a38076385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:01.708608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:01.733563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89d20a-ff71-4926-9c8e-4e2a38076385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:01.735014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89d20a-ff71-4926-9c8e-4e2a38076385 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:01.744509 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1344/2737] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:47:11.544520 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:11.778515 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=12 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:17.583908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f73fe3-00ef-4697-87a7-3a914ce5d132 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:17.665477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f73fe3-00ef-4697-87a7-3a914ce5d132 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:17.670384 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:17.671089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:17.671375 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:17.671644 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] PATH_INFO: `/v3/users/ca82ac2473d242e0b4244fec89cbbec8` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:17.672819 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca82ac2473d242e0b4244fec89cbbec8)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:17.674774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:17.966481 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89d20a-ff71-4926-9c8e-4e2a38076385 service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:17.968826 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8c89d20a-ff71-4926-9c8e-4e2a38076385 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:17.972767 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca82ac2473d242e0b4244fec89cbbec8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:17.985394 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89d20a-ff71-4926-9c8e-4e2a38076385 service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:17.990436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:17.991105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:17.991377 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:17.991660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] PATH_INFO: `/v3/users/cdc5e36c0c7745278eeecacc569c03dd` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:17.993372 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc5e36c0c7745278eeecacc569c03dd)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:17.994915 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:18.074620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca82ac2473d242e0b4244fec89cbbec8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.089820 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca82ac2473d242e0b4244fec89cbbec8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.090585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca82ac2473d242e0b4244fec89cbbec8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.099141 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-82a566b0-c5ec-4ff3-b4f5-03e70b956e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca82ac2473d242e0b4244fec89cbbec8'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.106028 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1394/2738] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:47:17 2022] DELETE /identity/v3/users/ca82ac2473d242e0b4244fec89cbbec8 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:18.111433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3e7b4-240d-4168-a4d0-dd51a3044f33 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:18.114519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3e7b4-240d-4168-a4d0-dd51a3044f33 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:18.114519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3e7b4-240d-4168-a4d0-dd51a3044f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:18.135756 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-54e3e7b4-240d-4168-a4d0-dd51a3044f33 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:18.137010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-54e3e7b4-240d-4168-a4d0-dd51a3044f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:18.199593 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc5e36c0c7745278eeecacc569c03dd'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.200419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc5e36c0c7745278eeecacc569c03dd'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.201100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc5e36c0c7745278eeecacc569c03dd'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.206822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1395/2739] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:18.209247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc5e36c0c7745278eeecacc569c03dd'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.220694 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3e7b4-240d-4168-a4d0-dd51a3044f33 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:18.222184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-54e3e7b4-240d-4168-a4d0-dd51a3044f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:18.229879 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2e4be36b-ca95-43ab-8d96-f72139339716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc5e36c0c7745278eeecacc569c03dd'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.254113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:18.292434 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3e7b4-240d-4168-a4d0-dd51a3044f33 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:18.297114 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a009-5ec2-49ef-9252-df6b50e4b206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:18.298320 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a009-5ec2-49ef-9252-df6b50e4b206 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:18.298590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a009-5ec2-49ef-9252-df6b50e4b206 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:18.298861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a009-5ec2-49ef-9252-df6b50e4b206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:18.308837 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548a009-5ec2-49ef-9252-df6b50e4b206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:18.310871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1345/2740] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:47:17 2022] DELETE /identity/v3/users/cdc5e36c0c7745278eeecacc569c03dd => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:18.311139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548a009-5ec2-49ef-9252-df6b50e4b206 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:18.315854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0899062-88ec-4f6c-a207-851ece88b53d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:18.316198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0899062-88ec-4f6c-a207-851ece88b53d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:18.316476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0899062-88ec-4f6c-a207-851ece88b53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:18.321753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1396/2741] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:47:18.335560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-a0899062-88ec-4f6c-a207-851ece88b53d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:18.336817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a0899062-88ec-4f6c-a207-851ece88b53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:18.408383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1346/2742] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:18.419287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a009-5ec2-49ef-9252-df6b50e4b206 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:18.421701 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b548a009-5ec2-49ef-9252-df6b50e4b206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:18.434022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a009-5ec2-49ef-9252-df6b50e4b206 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:18.489495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abd3b2-4186-4928-b48f-b464bf6fef98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:18.490110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abd3b2-4186-4928-b48f-b464bf6fef98 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:18.490389 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abd3b2-4186-4928-b48f-b464bf6fef98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:18.490659 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abd3b2-4186-4928-b48f-b464bf6fef98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:18.500839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abd3b2-4186-4928-b48f-b464bf6fef98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:18.502585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abd3b2-4186-4928-b48f-b464bf6fef98 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:18.512717 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1397/2743] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:47:18.783255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0899062-88ec-4f6c-a207-851ece88b53d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:18.784809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-a0899062-88ec-4f6c-a207-851ece88b53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:18.798155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0899062-88ec-4f6c-a207-851ece88b53d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:18.803044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:18.803698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:18.803989 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:18.804399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] PATH_INFO: `/v3/projects/edb50adb174e4348aea40289a23b4c18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:18.805627 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb50adb174e4348aea40289a23b4c18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:18.807554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:18.909953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb50adb174e4348aea40289a23b4c18', 'request_id': 'req-fb96e14e-baf1-4eaf-84a1-020951251aa9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.916075 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb50adb174e4348aea40289a23b4c18', 'request_id': 'req-fb96e14e-baf1-4eaf-84a1-020951251aa9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:18.996410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abd3b2-4186-4928-b48f-b464bf6fef98 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:18.998144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-c9abd3b2-4186-4928-b48f-b464bf6fef98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:19.011703 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abd3b2-4186-4928-b48f-b464bf6fef98 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:19.020934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb50adb174e4348aea40289a23b4c18', 'request_id': 'req-fb96e14e-baf1-4eaf-84a1-020951251aa9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.024178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:19.037549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] The token cache is being invalidate because project edb50adb174e4348aea40289a23b4c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:47:19.037851 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.038726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.040201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1347/2744] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:47:18 2022] DELETE /identity/v3/projects/edb50adb174e4348aea40289a23b4c18 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:19.044581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:19.046080 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:19.093318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:19.093971 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:19.094245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:19.094514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] PATH_INFO: `/v3/projects/af412f93b9ad4198b7fbfc3bf5863318` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:19.095824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=af412f93b9ad4198b7fbfc3bf5863318)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:19.098105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:19.131086 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96e14e-baf1-4eaf-84a1-020951251aa9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:19.137050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:19.141539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:19.141539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:19.141539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] PATH_INFO: `/v3/users/a3f1ada700fe4c77a69dc01761668a54` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:19.141539 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f1ada700fe4c77a69dc01761668a54)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:19.142503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:19.248010 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af412f93b9ad4198b7fbfc3bf5863318', 'request_id': 'req-10379ff0-cace-47b3-8a61-c4716d65ce1c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.255784 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af412f93b9ad4198b7fbfc3bf5863318', 'request_id': 'req-10379ff0-cace-47b3-8a61-c4716d65ce1c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.349959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f1ada700fe4c77a69dc01761668a54'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.350698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f1ada700fe4c77a69dc01761668a54'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.351475 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f1ada700fe4c77a69dc01761668a54'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.359571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f1ada700fe4c77a69dc01761668a54'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.363875 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af412f93b9ad4198b7fbfc3bf5863318', 'request_id': 'req-10379ff0-cace-47b3-8a61-c4716d65ce1c'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.369806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:19.373558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f1ada700fe4c77a69dc01761668a54'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.377128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] The token cache is being invalidate because project af412f93b9ad4198b7fbfc3bf5863318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:47:19.377423 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.378127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-10379ff0-cace-47b3-8a61-c4716d65ce1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:19.379678 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1398/2745] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:47:18 2022] DELETE /identity/v3/projects/af412f93b9ad4198b7fbfc3bf5863318 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:47:19.384597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a67a2-90b4-46fb-8ee3-1ffbe3d1b0e4 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:19.384929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a67a2-90b4-46fb-8ee3-1ffbe3d1b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:19.385397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a67a2-90b4-46fb-8ee3-1ffbe3d1b0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:19.406400 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-311a67a2-90b4-46fb-8ee3-1ffbe3d1b0e4 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:19.407637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-311a67a2-90b4-46fb-8ee3-1ffbe3d1b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:19.470930 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1399/2746] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:19.478616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63f6c-8ae4-405d-ad95-c5883d883f3f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:19.478896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63f6c-8ae4-405d-ad95-c5883d883f3f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:19.479180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f63f6c-8ae4-405d-ad95-c5883d883f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:19.499963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b1f63f6c-8ae4-405d-ad95-c5883d883f3f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:19.501600 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b1f63f6c-8ae4-405d-ad95-c5883d883f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:19.511848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1348/2747] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:47:19 2022] DELETE /identity/v3/users/a3f1ada700fe4c77a69dc01761668a54 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:47:19.511989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1400/2748] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:19.516510 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:19.517966 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:19.597828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b4466-d4d8-4724-ac1e-2407be5c789e admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:19.602755 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542625a-e521-410d-bc9a-9d88bb12eacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:19.603396 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542625a-e521-410d-bc9a-9d88bb12eacc admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:19.603667 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542625a-e521-410d-bc9a-9d88bb12eacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:19.603932 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542625a-e521-410d-bc9a-9d88bb12eacc admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:19.604577 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542625a-e521-410d-bc9a-9d88bb12eacc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:47:19.604951 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542625a-e521-410d-bc9a-9d88bb12eacc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:19.607215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542625a-e521-410d-bc9a-9d88bb12eacc admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:19.620081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1349/2749] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:47:19.627104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63f6c-8ae4-405d-ad95-c5883d883f3f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:19.639125 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f63f6c-8ae4-405d-ad95-c5883d883f3f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:19.644703 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f4e32-7c49-4771-a43c-92ca3498712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:19.645664 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f4e32-7c49-4771-a43c-92ca3498712e admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:19.646069 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f4e32-7c49-4771-a43c-92ca3498712e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:19.646469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f4e32-7c49-4771-a43c-92ca3498712e admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:19.647413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f4e32-7c49-4771-a43c-92ca3498712e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:19.650618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f4e32-7c49-4771-a43c-92ca3498712e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:19.775325 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1401/2750] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:47:19 2022] POST /identity/v3/projects => generated 359 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:47:19.783885 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542625a-e521-410d-bc9a-9d88bb12eacc admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:19.794625 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542625a-e521-410d-bc9a-9d88bb12eacc admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:19.799903 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:19.875010 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a78775-d35f-4d22-a4e7-8fa1ff6c33a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:19.876094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a78775-d35f-4d22-a4e7-8fa1ff6c33a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:19.876743 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a78775-d35f-4d22-a4e7-8fa1ff6c33a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:19.877158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a78775-d35f-4d22-a4e7-8fa1ff6c33a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:19.878617 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a78775-d35f-4d22-a4e7-8fa1ff6c33a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:19.880984 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a78775-d35f-4d22-a4e7-8fa1ff6c33a8 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:19.994259 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1350/2751] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:47:19 2022] POST /identity/v3/users => generated 280 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:47:19.995358 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f4e32-7c49-4771-a43c-92ca3498712e admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.008988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f4e32-7c49-4771-a43c-92ca3498712e admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.013685 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d378987-14bd-4f15-aafa-e1c9c5b47544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.014348 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d378987-14bd-4f15-aafa-e1c9c5b47544 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.014627 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d378987-14bd-4f15-aafa-e1c9c5b47544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.014901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d378987-14bd-4f15-aafa-e1c9c5b47544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.015546 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d378987-14bd-4f15-aafa-e1c9c5b47544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.017173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d378987-14bd-4f15-aafa-e1c9c5b47544 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.028035 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1402/2752] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:47:20.032846 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a78775-d35f-4d22-a4e7-8fa1ff6c33a8 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.043901 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a78775-d35f-4d22-a4e7-8fa1ff6c33a8 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.051075 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f69b9-047a-4e1e-8a86-4fc9977c887a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.054726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f69b9-047a-4e1e-8a86-4fc9977c887a admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.054726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f69b9-047a-4e1e-8a86-4fc9977c887a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.054726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f69b9-047a-4e1e-8a86-4fc9977c887a admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.054726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f69b9-047a-4e1e-8a86-4fc9977c887a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.054726 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f69b9-047a-4e1e-8a86-4fc9977c887a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.057870 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d378987-14bd-4f15-aafa-e1c9c5b47544 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.060261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9d378987-14bd-4f15-aafa-e1c9c5b47544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:20.076238 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1351/2753] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:47:20.078285 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d378987-14bd-4f15-aafa-e1c9c5b47544 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.088054 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f69b9-047a-4e1e-8a86-4fc9977c887a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.098698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.099276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.099692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.099807 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] PATH_INFO: `/v3/projects/bfc60d7559894255998f9c431aa1e61b` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.101303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc60d7559894255998f9c431aa1e61b)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.103604 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.103809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f69b9-047a-4e1e-8a86-4fc9977c887a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.110191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37542f5c-6c9d-4727-aea4-27bc941b98e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.111078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37542f5c-6c9d-4727-aea4-27bc941b98e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.111353 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37542f5c-6c9d-4727-aea4-27bc941b98e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.111798 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37542f5c-6c9d-4727-aea4-27bc941b98e0 admin admin] PATH_INFO: `/v3/projects/a27f9dae52a54ece8c73cee46e141096/users/6e274e208daf421e90746f9567975079/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.129377 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37542f5c-6c9d-4727-aea4-27bc941b98e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a27f9dae52a54ece8c73cee46e141096, user_id=6e274e208daf421e90746f9567975079, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.139848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37542f5c-6c9d-4727-aea4-27bc941b98e0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.221020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1352/2754] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:20 2022] PUT /identity/v3/projects/a27f9dae52a54ece8c73cee46e141096/users/6e274e208daf421e90746f9567975079/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:20.233503 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f57ad-20ee-44d5-90af-1574a5228506 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.233960 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f57ad-20ee-44d5-90af-1574a5228506 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.234367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f57ad-20ee-44d5-90af-1574a5228506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.238771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc60d7559894255998f9c431aa1e61b', 'request_id': 'req-04ea4d46-0990-41fa-bf55-bf19127c76b4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:20.250806 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc60d7559894255998f9c431aa1e61b', 'request_id': 'req-04ea4d46-0990-41fa-bf55-bf19127c76b4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:20.271744 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e06f57ad-20ee-44d5-90af-1574a5228506 None None] MFA Rules not processed for user `6e274e208daf421e90746f9567975079`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:20.272921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e06f57ad-20ee-44d5-90af-1574a5228506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:20.351965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1353/2755] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:20.373954 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc60d7559894255998f9c431aa1e61b', 'request_id': 'req-04ea4d46-0990-41fa-bf55-bf19127c76b4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:20.387383 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] The token cache is being invalidate because project bfc60d7559894255998f9c431aa1e61b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:47:20.388181 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:20.388864 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:20.390519 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1403/2756] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:47:20 2022] DELETE /identity/v3/projects/bfc60d7559894255998f9c431aa1e61b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:47:20.396492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb6b7c-84f6-4c82-b55b-39ab89cbc0f7 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.396982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb6b7c-84f6-4c82-b55b-39ab89cbc0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.397415 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb6b7c-84f6-4c82-b55b-39ab89cbc0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.397950 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.399495 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:20.423524 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-1afb6b7c-84f6-4c82-b55b-39ab89cbc0f7 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:20.425057 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-1afb6b7c-84f6-4c82-b55b-39ab89cbc0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:20.451632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:20.466237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4d46-0990-41fa-bf55-bf19127c76b4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.470771 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.473219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.473219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.473219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.474506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:20.478661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:20.486005 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.487503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.491559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1354/2757] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:20.497105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1404/2758] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:20 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:20.500184 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5a847-c75b-4b32-b9f6-442af998865e None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.500468 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5a847-c75b-4b32-b9f6-442af998865e None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.500741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5a847-c75b-4b32-b9f6-442af998865e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.513195 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.519055 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:20.521384 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-2ec5a847-c75b-4b32-b9f6-442af998865e None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:20.522620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2ec5a847-c75b-4b32-b9f6-442af998865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:20.529472 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55e768-facd-40fb-a392-c5b126d77ba3 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.531952 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1355/2759] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:20.536230 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5a847-c75b-4b32-b9f6-442af998865e None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.537418 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.538063 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.538340 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.538610 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] PATH_INFO: `/v3/users/6e274e208daf421e90746f9567975079` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.539776 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e274e208daf421e90746f9567975079)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.541749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.547641 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5a847-c75b-4b32-b9f6-442af998865e None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.552467 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27700ec4-2262-4ae0-b629-665df0fb6ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.553155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700ec4-2262-4ae0-b629-665df0fb6ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.553397 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700ec4-2262-4ae0-b629-665df0fb6ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.553682 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27700ec4-2262-4ae0-b629-665df0fb6ba6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.554348 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27700ec4-2262-4ae0-b629-665df0fb6ba6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:47:20.554655 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27700ec4-2262-4ae0-b629-665df0fb6ba6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.556268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27700ec4-2262-4ae0-b629-665df0fb6ba6 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.570779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1356/2760] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:47:20.575247 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27700ec4-2262-4ae0-b629-665df0fb6ba6 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.585106 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27700ec4-2262-4ae0-b629-665df0fb6ba6 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.589784 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93f33-cc31-461a-8c5b-09afa3a73976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.590395 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93f33-cc31-461a-8c5b-09afa3a73976 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.590808 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93f33-cc31-461a-8c5b-09afa3a73976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.591219 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc93f33-cc31-461a-8c5b-09afa3a73976 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.591876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc93f33-cc31-461a-8c5b-09afa3a73976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.593866 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc93f33-cc31-461a-8c5b-09afa3a73976 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.751870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1357/2761] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:47:20 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:47:20.756105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93f33-cc31-461a-8c5b-09afa3a73976 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.820502 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc93f33-cc31-461a-8c5b-09afa3a73976 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.825741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564e045-e2c6-4b84-ab45-0aff7896aa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.826364 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564e045-e2c6-4b84-ab45-0aff7896aa16 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.826642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564e045-e2c6-4b84-ab45-0aff7896aa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.826913 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564e045-e2c6-4b84-ab45-0aff7896aa16 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.827576 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564e045-e2c6-4b84-ab45-0aff7896aa16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.829554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564e045-e2c6-4b84-ab45-0aff7896aa16 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.856096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e274e208daf421e90746f9567975079'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:20.951551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1358/2762] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:47:20 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:47:20.960462 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564e045-e2c6-4b84-ab45-0aff7896aa16 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:20.972476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564e045-e2c6-4b84-ab45-0aff7896aa16 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:20.972981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e274e208daf421e90746f9567975079'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:20.980709 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af1130-1570-4fa2-97ff-3834ac7fcda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:20.982325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af1130-1570-4fa2-97ff-3834ac7fcda3 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:20.982771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af1130-1570-4fa2-97ff-3834ac7fcda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:20.983378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af1130-1570-4fa2-97ff-3834ac7fcda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:20.984273 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af1130-1570-4fa2-97ff-3834ac7fcda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:20.987099 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af1130-1570-4fa2-97ff-3834ac7fcda3 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:20.991833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e274e208daf421e90746f9567975079'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:20.992589 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e274e208daf421e90746f9567975079'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:20.999441 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1359/2763] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:47:21.001579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-afbea436-f6bc-4e9f-8bb3-5f2beafb07f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e274e208daf421e90746f9567975079'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:21.003685 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af1130-1570-4fa2-97ff-3834ac7fcda3 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:21.005494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b5af1130-1570-4fa2-97ff-3834ac7fcda3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:21.011251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1405/2764] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:47:20 2022] DELETE /identity/v3/users/6e274e208daf421e90746f9567975079 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:21.016243 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f50279-5d06-43c8-9d23-8eb9abf80bfc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.016502 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f50279-5d06-43c8-9d23-8eb9abf80bfc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.016774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f50279-5d06-43c8-9d23-8eb9abf80bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.022666 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af1130-1570-4fa2-97ff-3834ac7fcda3 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:21.038521 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-66f50279-5d06-43c8-9d23-8eb9abf80bfc None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:21.040430 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-66f50279-5d06-43c8-9d23-8eb9abf80bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:21.091178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a921f-2ae6-4f6d-a3be-bc0dc1d06d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:21.091736 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a921f-2ae6-4f6d-a3be-bc0dc1d06d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.092078 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a921f-2ae6-4f6d-a3be-bc0dc1d06d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.092352 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a921f-2ae6-4f6d-a3be-bc0dc1d06d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.093026 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a921f-2ae6-4f6d-a3be-bc0dc1d06d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:21.094637 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a921f-2ae6-4f6d-a3be-bc0dc1d06d2d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:21.104988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1406/2765] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:21.112882 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1360/2766] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:47:21.115661 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a921f-2ae6-4f6d-a3be-bc0dc1d06d2d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:21.115799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f50279-5d06-43c8-9d23-8eb9abf80bfc None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:21.117074 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-66f50279-5d06-43c8-9d23-8eb9abf80bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:21.126574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a921f-2ae6-4f6d-a3be-bc0dc1d06d2d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:21.131631 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edeaea-2848-473a-90ca-757e272c3579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:21.132293 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edeaea-2848-473a-90ca-757e272c3579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.132931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edeaea-2848-473a-90ca-757e272c3579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.133207 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edeaea-2848-473a-90ca-757e272c3579 admin admin] PATH_INFO: `/v3/projects/3741d7ba9a904c04a079c0a558250003/users/35c821fed3344b62ab3a3bcb1a2412b4/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.150729 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edeaea-2848-473a-90ca-757e272c3579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3741d7ba9a904c04a079c0a558250003, user_id=35c821fed3344b62ab3a3bcb1a2412b4, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:21.163059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edeaea-2848-473a-90ca-757e272c3579 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:21.187993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f50279-5d06-43c8-9d23-8eb9abf80bfc None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:21.191814 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9976dc8-e276-4fe4-9d90-873a5ca59492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:21.192360 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9976dc8-e276-4fe4-9d90-873a5ca59492 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.192637 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9976dc8-e276-4fe4-9d90-873a5ca59492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.192955 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9976dc8-e276-4fe4-9d90-873a5ca59492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.203453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9976dc8-e276-4fe4-9d90-873a5ca59492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:21.204968 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9976dc8-e276-4fe4-9d90-873a5ca59492 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:21.215269 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:21.216536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1407/2767] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:21.282724 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1361/2768] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:21 2022] PUT /identity/v3/projects/3741d7ba9a904c04a079c0a558250003/users/35c821fed3344b62ab3a3bcb1a2412b4/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:21.358130 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea5bf3-58a1-4df2-956a-fed4d558969c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.358552 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea5bf3-58a1-4df2-956a-fed4d558969c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.358827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea5bf3-58a1-4df2-956a-fed4d558969c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.389793 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-07ea5bf3-58a1-4df2-956a-fed4d558969c None None] MFA Rules not processed for user `35c821fed3344b62ab3a3bcb1a2412b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:21.390995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-07ea5bf3-58a1-4df2-956a-fed4d558969c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:21.456104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1408/2769] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:21.468107 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edeaea-2848-473a-90ca-757e272c3579 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:21.481560 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:21.548191 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edeaea-2848-473a-90ca-757e272c3579 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:21.553132 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584a7da-a373-45f3-8750-481f78341155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:21.553595 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584a7da-a373-45f3-8750-481f78341155 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.553868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584a7da-a373-45f3-8750-481f78341155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.554134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584a7da-a373-45f3-8750-481f78341155 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.554933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584a7da-a373-45f3-8750-481f78341155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:21.556642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584a7da-a373-45f3-8750-481f78341155 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:21.657818 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1362/2770] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:47:21 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:47:21.661726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea5bf3-58a1-4df2-956a-fed4d558969c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:21.676998 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea5bf3-58a1-4df2-956a-fed4d558969c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:21.704347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584a7da-a373-45f3-8750-481f78341155 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:21.706792 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0584a7da-a373-45f3-8750-481f78341155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:21.745853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86d6b4-e857-4784-8928-bd26e18445db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:21.746462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86d6b4-e857-4784-8928-bd26e18445db admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.746735 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86d6b4-e857-4784-8928-bd26e18445db admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.747660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86d6b4-e857-4784-8928-bd26e18445db admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.748438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86d6b4-e857-4784-8928-bd26e18445db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:21.751231 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86d6b4-e857-4784-8928-bd26e18445db admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:21.789500 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584a7da-a373-45f3-8750-481f78341155 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:21.794360 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:21.800497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.800497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.800497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] PATH_INFO: `/v3/projects/a27f9dae52a54ece8c73cee46e141096` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.800497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a27f9dae52a54ece8c73cee46e141096)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:21.800497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:21.859135 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1409/2771] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:47:21 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:47:21.863024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86d6b4-e857-4784-8928-bd26e18445db admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:21.874401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86d6b4-e857-4784-8928-bd26e18445db admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:21.879323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf98d7-8ba4-4a0b-a1b2-73ae1ffff290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:21.879906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf98d7-8ba4-4a0b-a1b2-73ae1ffff290 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.880182 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf98d7-8ba4-4a0b-a1b2-73ae1ffff290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.880451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf98d7-8ba4-4a0b-a1b2-73ae1ffff290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.881119 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf98d7-8ba4-4a0b-a1b2-73ae1ffff290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:21.882738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf98d7-8ba4-4a0b-a1b2-73ae1ffff290 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:21.893376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1410/2772] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:47:21.898774 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf98d7-8ba4-4a0b-a1b2-73ae1ffff290 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:21.903954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a27f9dae52a54ece8c73cee46e141096', 'request_id': 'req-f4a904de-7b64-44c6-92ee-b95f2d5527e9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:21.909292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf98d7-8ba4-4a0b-a1b2-73ae1ffff290 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:21.910261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a27f9dae52a54ece8c73cee46e141096', 'request_id': 'req-f4a904de-7b64-44c6-92ee-b95f2d5527e9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:21.966994 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fb9a7-ff6d-4120-884e-eaffd574f3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:21.967602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fb9a7-ff6d-4120-884e-eaffd574f3af admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:21.967907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fb9a7-ff6d-4120-884e-eaffd574f3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:21.968159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fb9a7-ff6d-4120-884e-eaffd574f3af admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:21.968788 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6fb9a7-ff6d-4120-884e-eaffd574f3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:21.970403 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6fb9a7-ff6d-4120-884e-eaffd574f3af admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:21.975060 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a27f9dae52a54ece8c73cee46e141096', 'request_id': 'req-f4a904de-7b64-44c6-92ee-b95f2d5527e9'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:21.982932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1411/2773] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:47:21.987151 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fb9a7-ff6d-4120-884e-eaffd574f3af admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:21.992726 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:22.009040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] The token cache is being invalidate because project a27f9dae52a54ece8c73cee46e141096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:47:22.009040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:22.009040 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f4a904de-7b64-44c6-92ee-b95f2d5527e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:22.009347 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fb9a7-ff6d-4120-884e-eaffd574f3af admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.009434 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1363/2774] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:47:21 2022] DELETE /identity/v3/projects/a27f9dae52a54ece8c73cee46e141096 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:22.009434 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588a68d-edcb-4b85-af0d-417f116c20f0 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.009434 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588a68d-edcb-4b85-af0d-417f116c20f0 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.009692 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86447bb5-10ee-4637-864b-b15098d3ae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.009810 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588a68d-edcb-4b85-af0d-417f116c20f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.009903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447bb5-10ee-4637-864b-b15098d3ae98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.009903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447bb5-10ee-4637-864b-b15098d3ae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.009903 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86447bb5-10ee-4637-864b-b15098d3ae98 admin admin] PATH_INFO: `/v3/projects/2d251076e7b4484d9f3500589b16d7df/users/4535aaa5a90540f083d5a1365aed0f58/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.025969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-7588a68d-edcb-4b85-af0d-417f116c20f0 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:22.032085 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86447bb5-10ee-4637-864b-b15098d3ae98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d251076e7b4484d9f3500589b16d7df, user_id=4535aaa5a90540f083d5a1365aed0f58, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.032565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-7588a68d-edcb-4b85-af0d-417f116c20f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:22.032672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86447bb5-10ee-4637-864b-b15098d3ae98 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.037278 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1364/2775] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:22.046366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aec603-362e-4267-b7d4-405c3e626e1d None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.046652 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aec603-362e-4267-b7d4-405c3e626e1d None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.046922 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aec603-362e-4267-b7d4-405c3e626e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.067314 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-00aec603-362e-4267-b7d4-405c3e626e1d None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:22.068781 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-00aec603-362e-4267-b7d4-405c3e626e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:22.076782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1365/2776] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:22.080908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aec603-362e-4267-b7d4-405c3e626e1d None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.086998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:22.090779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aec603-362e-4267-b7d4-405c3e626e1d None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.095771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c31b6-f29f-4b4a-a42e-9a7a2e5428e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.096362 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c31b6-f29f-4b4a-a42e-9a7a2e5428e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.096620 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c31b6-f29f-4b4a-a42e-9a7a2e5428e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.096925 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c31b6-f29f-4b4a-a42e-9a7a2e5428e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.097561 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c31b6-f29f-4b4a-a42e-9a7a2e5428e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:47:22.097859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c31b6-f29f-4b4a-a42e-9a7a2e5428e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.099436 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c31b6-f29f-4b4a-a42e-9a7a2e5428e9 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.111688 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1412/2777] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:21 2022] PUT /identity/v3/projects/2d251076e7b4484d9f3500589b16d7df/users/4535aaa5a90540f083d5a1365aed0f58/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:47:22.120354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1366/2778] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 11:47:22.120354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c31b6-f29f-4b4a-a42e-9a7a2e5428e9 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.122992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9a246-682c-45ad-a2ba-f63cde0b639c None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.123346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9a246-682c-45ad-a2ba-f63cde0b639c None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.123687 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9a246-682c-45ad-a2ba-f63cde0b639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.128198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c31b6-f29f-4b4a-a42e-9a7a2e5428e9 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.152674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-daf9a246-682c-45ad-a2ba-f63cde0b639c None None] MFA Rules not processed for user `4535aaa5a90540f083d5a1365aed0f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:22.154298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-daf9a246-682c-45ad-a2ba-f63cde0b639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:22.187707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f039f-0b6c-43e8-a86f-a929d15ff4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.188325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f039f-0b6c-43e8-a86f-a929d15ff4bb admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.191640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f039f-0b6c-43e8-a86f-a929d15ff4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.191640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f039f-0b6c-43e8-a86f-a929d15ff4bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.191640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f039f-0b6c-43e8-a86f-a929d15ff4bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.191640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f039f-0b6c-43e8-a86f-a929d15ff4bb admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.222942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1413/2779] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:22.227629 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9a246-682c-45ad-a2ba-f63cde0b639c None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.229159 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-daf9a246-682c-45ad-a2ba-f63cde0b639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:22.243022 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9a246-682c-45ad-a2ba-f63cde0b639c None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.248658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121d481-83e7-4a7e-88b4-6562bf8f7eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.249490 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121d481-83e7-4a7e-88b4-6562bf8f7eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.249860 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121d481-83e7-4a7e-88b4-6562bf8f7eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.250228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121d481-83e7-4a7e-88b4-6562bf8f7eb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.250963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121d481-83e7-4a7e-88b4-6562bf8f7eb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.253503 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121d481-83e7-4a7e-88b4-6562bf8f7eb6 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.334626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1367/2780] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:47:22 2022] POST /identity/v3/projects => generated 369 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:47:22.338094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f039f-0b6c-43e8-a86f-a929d15ff4bb admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.387841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1414/2781] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:47:22 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:47:22.392007 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121d481-83e7-4a7e-88b4-6562bf8f7eb6 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.399262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f039f-0b6c-43e8-a86f-a929d15ff4bb admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.465158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7909a1d-18f0-4dfe-8d72-dfea90facc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.466299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7909a1d-18f0-4dfe-8d72-dfea90facc28 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.466678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7909a1d-18f0-4dfe-8d72-dfea90facc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.467061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7909a1d-18f0-4dfe-8d72-dfea90facc28 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.467683 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7909a1d-18f0-4dfe-8d72-dfea90facc28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.469619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7909a1d-18f0-4dfe-8d72-dfea90facc28 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.475753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121d481-83e7-4a7e-88b4-6562bf8f7eb6 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.487643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a5aae-796f-438c-8ffa-2c6a65d7a764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.488322 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a5aae-796f-438c-8ffa-2c6a65d7a764 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.488608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a5aae-796f-438c-8ffa-2c6a65d7a764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.488908 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a5aae-796f-438c-8ffa-2c6a65d7a764 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.489529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a5aae-796f-438c-8ffa-2c6a65d7a764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.491496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a5aae-796f-438c-8ffa-2c6a65d7a764 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.595559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1415/2782] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:47:22 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:47:22.595982 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1368/2783] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:47:22 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:47:22.599757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7909a1d-18f0-4dfe-8d72-dfea90facc28 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.601673 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a5aae-796f-438c-8ffa-2c6a65d7a764 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.610643 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7909a1d-18f0-4dfe-8d72-dfea90facc28 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.612522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a5aae-796f-438c-8ffa-2c6a65d7a764 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.615336 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a3ccd-f82c-4726-8b21-afceeea3f0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.615930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a3ccd-f82c-4726-8b21-afceeea3f0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.616208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a3ccd-f82c-4726-8b21-afceeea3f0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.616474 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a3ccd-f82c-4726-8b21-afceeea3f0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.617221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a3ccd-f82c-4726-8b21-afceeea3f0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.617333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360992af-f43a-4407-bd01-13c52185849b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.617871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360992af-f43a-4407-bd01-13c52185849b admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.618142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360992af-f43a-4407-bd01-13c52185849b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.618411 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360992af-f43a-4407-bd01-13c52185849b admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.618921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a3ccd-f82c-4726-8b21-afceeea3f0d0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.619073 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360992af-f43a-4407-bd01-13c52185849b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.620662 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360992af-f43a-4407-bd01-13c52185849b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.630100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1369/2784] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:47:22.631713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1416/2785] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:47:22.639447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a3ccd-f82c-4726-8b21-afceeea3f0d0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.645077 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:22.651098 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a3ccd-f82c-4726-8b21-afceeea3f0d0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.658169 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204c53e-c222-4334-93d9-d68a62be4981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.659141 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204c53e-c222-4334-93d9-d68a62be4981 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.659590 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204c53e-c222-4334-93d9-d68a62be4981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.665898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204c53e-c222-4334-93d9-d68a62be4981 admin admin] PATH_INFO: `/v3/projects/8936d4d0ab9549628e7819c69211b56f/users/2f0fb264bd7a4901a32ed90933b3d2aa/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.681446 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204c53e-c222-4334-93d9-d68a62be4981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8936d4d0ab9549628e7819c69211b56f, user_id=2f0fb264bd7a4901a32ed90933b3d2aa, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.684203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204c53e-c222-4334-93d9-d68a62be4981 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.694420 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360992af-f43a-4407-bd01-13c52185849b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.705554 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360992af-f43a-4407-bd01-13c52185849b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.710498 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5b18e-3492-4fc5-a8e7-be3a6db0fbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.711094 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5b18e-3492-4fc5-a8e7-be3a6db0fbb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.711371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5b18e-3492-4fc5-a8e7-be3a6db0fbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.711642 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5b18e-3492-4fc5-a8e7-be3a6db0fbb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.712280 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5b18e-3492-4fc5-a8e7-be3a6db0fbb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.713907 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5b18e-3492-4fc5-a8e7-be3a6db0fbb1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.726202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1417/2786] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:47:22.730197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5b18e-3492-4fc5-a8e7-be3a6db0fbb1 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.740201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5b18e-3492-4fc5-a8e7-be3a6db0fbb1 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.744880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b89dc6-81b2-4a9e-bb60-1158ad34b10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.745517 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b89dc6-81b2-4a9e-bb60-1158ad34b10e admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.745794 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b89dc6-81b2-4a9e-bb60-1158ad34b10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.746067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b89dc6-81b2-4a9e-bb60-1158ad34b10e admin admin] PATH_INFO: `/v3/projects/c3aca5bfdf474fc7a7ddb0f9fc0d5e2f/users/6bbb460842f44f4d81acb507f04469fe/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.759170 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:22.763719 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b89dc6-81b2-4a9e-bb60-1158ad34b10e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3aca5bfdf474fc7a7ddb0f9fc0d5e2f, user_id=6bbb460842f44f4d81acb507f04469fe, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.766569 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b89dc6-81b2-4a9e-bb60-1158ad34b10e admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.775284 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1370/2787] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:22 2022] PUT /identity/v3/projects/8936d4d0ab9549628e7819c69211b56f/users/2f0fb264bd7a4901a32ed90933b3d2aa/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:22.779709 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204c53e-c222-4334-93d9-d68a62be4981 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.841884 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204c53e-c222-4334-93d9-d68a62be4981 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.846771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa9fe3-cee4-4d75-bbba-dd7cbdaa202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:22.847378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa9fe3-cee4-4d75-bbba-dd7cbdaa202d admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.847657 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa9fe3-cee4-4d75-bbba-dd7cbdaa202d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.847964 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa9fe3-cee4-4d75-bbba-dd7cbdaa202d admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.848598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa9fe3-cee4-4d75-bbba-dd7cbdaa202d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:22.849937 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1418/2788] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:22 2022] PUT /identity/v3/projects/c3aca5bfdf474fc7a7ddb0f9fc0d5e2f/users/6bbb460842f44f4d81acb507f04469fe/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:47:22.850289 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa9fe3-cee4-4d75-bbba-dd7cbdaa202d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:22.860161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89515a8a-6bcf-404b-af5d-fe62634887d0 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:22.860741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89515a8a-6bcf-404b-af5d-fe62634887d0 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:22.860741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89515a8a-6bcf-404b-af5d-fe62634887d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:22.861036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1371/2789] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:47:22.865064 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa9fe3-cee4-4d75-bbba-dd7cbdaa202d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.876017 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa9fe3-cee4-4d75-bbba-dd7cbdaa202d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:22.891086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-89515a8a-6bcf-404b-af5d-fe62634887d0 None None] MFA Rules not processed for user `6bbb460842f44f4d81acb507f04469fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:22.892437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-89515a8a-6bcf-404b-af5d-fe62634887d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:22.964909 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1419/2790] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:22.978728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89515a8a-6bcf-404b-af5d-fe62634887d0 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:22.980155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-89515a8a-6bcf-404b-af5d-fe62634887d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:23.052139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89515a8a-6bcf-404b-af5d-fe62634887d0 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:23.056054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f977878-9dd1-4931-a12c-da798c538c42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ac781cf2d84d95a789724e74e12acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:23.056656 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f977878-9dd1-4931-a12c-da798c538c42 service glance] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:23.056962 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f977878-9dd1-4931-a12c-da798c538c42 service glance] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:23.057233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f977878-9dd1-4931-a12c-da798c538c42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:23.062549 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460a4af-9462-45da-aa80-39a733c73769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:23.063205 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460a4af-9462-45da-aa80-39a733c73769 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:23.063478 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460a4af-9462-45da-aa80-39a733c73769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:23.063748 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460a4af-9462-45da-aa80-39a733c73769 admin admin] PATH_INFO: `/v3/projects/8936d4d0ab9549628e7819c69211b56f/users/2f0fb264bd7a4901a32ed90933b3d2aa/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:23.065803 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460a4af-9462-45da-aa80-39a733c73769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8936d4d0ab9549628e7819c69211b56f, user_id=2f0fb264bd7a4901a32ed90933b3d2aa, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:23.066674 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f977878-9dd1-4931-a12c-da798c538c42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:23.068202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f977878-9dd1-4931-a12c-da798c538c42 service glance] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:23.068553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460a4af-9462-45da-aa80-39a733c73769 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:23.078054 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1420/2791] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:47:23.080582 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1372/2792] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:22 2022] PUT /identity/v3/projects/8936d4d0ab9549628e7819c69211b56f/users/2f0fb264bd7a4901a32ed90933b3d2aa/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:47:23.092401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be74b0-069b-40a6-89b1-4ff62e5335dc None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:23.092682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be74b0-069b-40a6-89b1-4ff62e5335dc None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:23.092989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be74b0-069b-40a6-89b1-4ff62e5335dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:23.124738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e7be74b0-069b-40a6-89b1-4ff62e5335dc None None] MFA Rules not processed for user `2f0fb264bd7a4901a32ed90933b3d2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:23.126004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e7be74b0-069b-40a6-89b1-4ff62e5335dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:23.202598 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1421/2793] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:23.214369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460a4af-9462-45da-aa80-39a733c73769 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:23.216302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-9460a4af-9462-45da-aa80-39a733c73769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:23.276379 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460a4af-9462-45da-aa80-39a733c73769 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:23.280363 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a30e38-79b0-4d3a-9330-20dda0b4e5c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:23.281011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a30e38-79b0-4d3a-9330-20dda0b4e5c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:23.283916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a30e38-79b0-4d3a-9330-20dda0b4e5c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:23.283916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a30e38-79b0-4d3a-9330-20dda0b4e5c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:23.283916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-34a30e38-79b0-4d3a-9330-20dda0b4e5c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:23.305457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:23.352979 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a30e38-79b0-4d3a-9330-20dda0b4e5c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:23.354758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a30e38-79b0-4d3a-9330-20dda0b4e5c6 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:23.375769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1373/2794] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:47:29.791965 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be74b0-069b-40a6-89b1-4ff62e5335dc None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:29.802649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be74b0-069b-40a6-89b1-4ff62e5335dc None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:29.806635 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7617119-ac96-49a0-9434-0a9e041166b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:29.807251 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7617119-ac96-49a0-9434-0a9e041166b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:29.807576 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7617119-ac96-49a0-9434-0a9e041166b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:29.807848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7617119-ac96-49a0-9434-0a9e041166b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:29.813863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:29.884021 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7617119-ac96-49a0-9434-0a9e041166b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:29.885677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7617119-ac96-49a0-9434-0a9e041166b6 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:29.900549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1422/2795] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:31.822090 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a30e38-79b0-4d3a-9330-20dda0b4e5c6 service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:31.832126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a30e38-79b0-4d3a-9330-20dda0b4e5c6 service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:31.836531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1001a-23b4-4700-8c50-566059da59a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:31.837201 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1001a-23b4-4700-8c50-566059da59a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:31.837920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1001a-23b4-4700-8c50-566059da59a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:31.837920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1001a-23b4-4700-8c50-566059da59a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:31.848354 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1001a-23b4-4700-8c50-566059da59a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:31.849943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1001a-23b4-4700-8c50-566059da59a1 service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:31.862244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1374/2796] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:36.540370 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7617119-ac96-49a0-9434-0a9e041166b6 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:36.541942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f7617119-ac96-49a0-9434-0a9e041166b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:36.623205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7617119-ac96-49a0-9434-0a9e041166b6 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:36.628219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:36.628871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:36.629144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:36.629414 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] PATH_INFO: `/v3/users/4bda959c0c0e406d9a02fbeb53fb2db2` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:36.630581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bda959c0c0e406d9a02fbeb53fb2db2)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:36.632504 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:36.962514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bda959c0c0e406d9a02fbeb53fb2db2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:37.115048 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bda959c0c0e406d9a02fbeb53fb2db2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:37.129660 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bda959c0c0e406d9a02fbeb53fb2db2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:37.130690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bda959c0c0e406d9a02fbeb53fb2db2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:37.140451 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bda959c0c0e406d9a02fbeb53fb2db2'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:37.146608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1423/2797] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:47:36 2022] DELETE /identity/v3/users/4bda959c0c0e406d9a02fbeb53fb2db2 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:37.151130 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f80d2-111b-4a3f-8a5e-a797e3c86ebc None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:37.151660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f80d2-111b-4a3f-8a5e-a797e3c86ebc None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:37.151660 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f80d2-111b-4a3f-8a5e-a797e3c86ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:37.184569 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-953f80d2-111b-4a3f-8a5e-a797e3c86ebc None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:37.185868 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-953f80d2-111b-4a3f-8a5e-a797e3c86ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:37.268993 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1375/2798] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:37.279485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:37.281050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:37.321032 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:37.353307 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e683b-8ecc-4ae9-b9a8-2e12f73a215d admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:37.357086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37185476-eddb-4ad6-adb2-2e85b1d78ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:37.357789 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37185476-eddb-4ad6-adb2-2e85b1d78ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:37.358070 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37185476-eddb-4ad6-adb2-2e85b1d78ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:37.358354 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37185476-eddb-4ad6-adb2-2e85b1d78ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:37.368174 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37185476-eddb-4ad6-adb2-2e85b1d78ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:37.369931 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37185476-eddb-4ad6-adb2-2e85b1d78ba2 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:37.381594 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1424/2799] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:47:37.946274 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f80d2-111b-4a3f-8a5e-a797e3c86ebc None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:37.949758 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-953f80d2-111b-4a3f-8a5e-a797e3c86ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:37.975272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f80d2-111b-4a3f-8a5e-a797e3c86ebc None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:37.981514 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:37.982158 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:37.982434 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:37.982833 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] PATH_INFO: `/v3/projects/a40fb9a256b4483a85ea730c48621070` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:37.984174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a40fb9a256b4483a85ea730c48621070)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:37.986214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:38.162973 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37185476-eddb-4ad6-adb2-2e85b1d78ba2 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:38.166608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-37185476-eddb-4ad6-adb2-2e85b1d78ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:38.182025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37185476-eddb-4ad6-adb2-2e85b1d78ba2 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:38.196068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a40fb9a256b4483a85ea730c48621070', 'request_id': 'req-1bd562a1-95f4-488d-bd7f-12afa3c4b433'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.202566 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a40fb9a256b4483a85ea730c48621070', 'request_id': 'req-1bd562a1-95f4-488d-bd7f-12afa3c4b433'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.249749 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:38.250691 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.251278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.251407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] PATH_INFO: `/v3/users/35c821fed3344b62ab3a3bcb1a2412b4` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.253165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c821fed3344b62ab3a3bcb1a2412b4)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:38.255252 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:38.301025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a40fb9a256b4483a85ea730c48621070', 'request_id': 'req-1bd562a1-95f4-488d-bd7f-12afa3c4b433'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.314105 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] The token cache is being invalidate because project a40fb9a256b4483a85ea730c48621070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:47:38.314407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.315160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-1bd562a1-95f4-488d-bd7f-12afa3c4b433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.316854 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1376/2800] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:47:37 2022] DELETE /identity/v3/projects/a40fb9a256b4483a85ea730c48621070 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:38.325397 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f70d63-cd7e-4bc3-a58f-da7af069641b None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.325746 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f70d63-cd7e-4bc3-a58f-da7af069641b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.325931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f70d63-cd7e-4bc3-a58f-da7af069641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.353178 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-33f70d63-cd7e-4bc3-a58f-da7af069641b None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:38.354302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-33f70d63-cd7e-4bc3-a58f-da7af069641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:38.363809 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1377/2801] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:38.376943 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ab699-4896-4489-83c8-65e1fd781841 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.377533 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ab699-4896-4489-83c8-65e1fd781841 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.377836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ab699-4896-4489-83c8-65e1fd781841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.400706 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-978ab699-4896-4489-83c8-65e1fd781841 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:38.403811 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-978ab699-4896-4489-83c8-65e1fd781841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:38.417325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1378/2802] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:38.424125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ab699-4896-4489-83c8-65e1fd781841 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:38.434025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ab699-4896-4489-83c8-65e1fd781841 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:38.439392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9decc-94b8-4c08-859a-6fd00377d0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:38.440023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9decc-94b8-4c08-859a-6fd00377d0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.440303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9decc-94b8-4c08-859a-6fd00377d0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.440575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9decc-94b8-4c08-859a-6fd00377d0c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.441266 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9decc-94b8-4c08-859a-6fd00377d0c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:47:38.441565 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9decc-94b8-4c08-859a-6fd00377d0c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:38.443262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9decc-94b8-4c08-859a-6fd00377d0c7 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:38.458115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1379/2803] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:47:38.462889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9decc-94b8-4c08-859a-6fd00377d0c7 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:38.475495 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9decc-94b8-4c08-859a-6fd00377d0c7 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:38.480865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5a2b2-286a-4a8f-9579-8941e4c674a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:38.481608 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5a2b2-286a-4a8f-9579-8941e4c674a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.481965 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5a2b2-286a-4a8f-9579-8941e4c674a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.482292 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5a2b2-286a-4a8f-9579-8941e4c674a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.482977 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5a2b2-286a-4a8f-9579-8941e4c674a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:38.484955 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5a2b2-286a-4a8f-9579-8941e4c674a0 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:38.492630 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:38.557974 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c821fed3344b62ab3a3bcb1a2412b4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.651984 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c821fed3344b62ab3a3bcb1a2412b4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.663911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1380/2804] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:47:38 2022] POST /identity/v3/projects => generated 389 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:47:38.667364 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c821fed3344b62ab3a3bcb1a2412b4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.669212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5a2b2-286a-4a8f-9579-8941e4c674a0 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:38.669946 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c821fed3344b62ab3a3bcb1a2412b4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.678207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-12fc49f7-354d-4343-970c-102eaa39f4c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c821fed3344b62ab3a3bcb1a2412b4'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:38.685431 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1425/2805] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:47:38 2022] DELETE /identity/v3/users/35c821fed3344b62ab3a3bcb1a2412b4 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:47:38.689753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0021f-16b0-45b6-add6-3b55f2c623fd None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.690132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0021f-16b0-45b6-add6-3b55f2c623fd None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.690365 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0021f-16b0-45b6-add6-3b55f2c623fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.710506 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-4bb0021f-16b0-45b6-add6-3b55f2c623fd None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:38.711822 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4bb0021f-16b0-45b6-add6-3b55f2c623fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:38.735640 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5a2b2-286a-4a8f-9579-8941e4c674a0 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:38.767089 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:38.786186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1426/2806] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:38.798019 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0021f-16b0-45b6-add6-3b55f2c623fd None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:38.799376 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4bb0021f-16b0-45b6-add6-3b55f2c623fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:38.807927 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f163ea-48a9-4d9a-976d-3c5bc8ee7914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:38.808531 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f163ea-48a9-4d9a-976d-3c5bc8ee7914 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.808930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f163ea-48a9-4d9a-976d-3c5bc8ee7914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.809203 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f163ea-48a9-4d9a-976d-3c5bc8ee7914 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.809828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f163ea-48a9-4d9a-976d-3c5bc8ee7914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:38.811852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f163ea-48a9-4d9a-976d-3c5bc8ee7914 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:38.866866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0021f-16b0-45b6-add6-3b55f2c623fd None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:38.870696 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db441579-0aaf-43b6-93d0-5738111e5b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:38.871866 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db441579-0aaf-43b6-93d0-5738111e5b60 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.872139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db441579-0aaf-43b6-93d0-5738111e5b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.872407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db441579-0aaf-43b6-93d0-5738111e5b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.882044 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db441579-0aaf-43b6-93d0-5738111e5b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:38.883493 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db441579-0aaf-43b6-93d0-5738111e5b60 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:38.894690 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1427/2807] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:47:38.914477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1381/2808] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:47:38 2022] POST /identity/v3/users => generated 295 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:47:38.915712 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db441579-0aaf-43b6-93d0-5738111e5b60 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:38.922097 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-db441579-0aaf-43b6-93d0-5738111e5b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:38.939415 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db441579-0aaf-43b6-93d0-5738111e5b60 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:38.944587 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a49cc-cfe7-456e-a266-1723136e79e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:38.951714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a49cc-cfe7-456e-a266-1723136e79e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.951714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a49cc-cfe7-456e-a266-1723136e79e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.951714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a49cc-cfe7-456e-a266-1723136e79e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.951714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a49cc-cfe7-456e-a266-1723136e79e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:38.951714 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a49cc-cfe7-456e-a266-1723136e79e5 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:38.959841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1428/2809] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:47:38.964814 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f163ea-48a9-4d9a-976d-3c5bc8ee7914 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:38.975494 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f163ea-48a9-4d9a-976d-3c5bc8ee7914 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:38.981212 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272916f-a56f-4b74-bbba-c197937ff572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:38.981843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272916f-a56f-4b74-bbba-c197937ff572 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:38.982418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272916f-a56f-4b74-bbba-c197937ff572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:38.982690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272916f-a56f-4b74-bbba-c197937ff572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:38.983331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272916f-a56f-4b74-bbba-c197937ff572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:38.984968 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272916f-a56f-4b74-bbba-c197937ff572 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:38.996718 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1382/2810] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:47:39.001201 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a49cc-cfe7-456e-a266-1723136e79e5 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:39.012036 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a49cc-cfe7-456e-a266-1723136e79e5 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:39.022647 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6319fb7-9c70-4369-852d-ad7af6da7d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:39.023461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6319fb7-9c70-4369-852d-ad7af6da7d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:39.023833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6319fb7-9c70-4369-852d-ad7af6da7d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:39.024104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6319fb7-9c70-4369-852d-ad7af6da7d74 admin admin] PATH_INFO: `/v3/projects/72bc32a0f8ad4a1fb4feb90edb84b3c0/users/7ac44c0375c04784b1fc10805b64a35f/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:39.043863 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6319fb7-9c70-4369-852d-ad7af6da7d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72bc32a0f8ad4a1fb4feb90edb84b3c0, user_id=7ac44c0375c04784b1fc10805b64a35f, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:39.047429 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6319fb7-9c70-4369-852d-ad7af6da7d74 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:39.211689 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1429/2811] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:39 2022] PUT /identity/v3/projects/72bc32a0f8ad4a1fb4feb90edb84b3c0/users/7ac44c0375c04784b1fc10805b64a35f/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:39.274958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77efabc-a470-460d-8db4-d7a6d1444aae None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:39.275848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77efabc-a470-460d-8db4-d7a6d1444aae None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:39.275848 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77efabc-a470-460d-8db4-d7a6d1444aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:39.311988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b77efabc-a470-460d-8db4-d7a6d1444aae None None] MFA Rules not processed for user `7ac44c0375c04784b1fc10805b64a35f`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:39.313823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b77efabc-a470-460d-8db4-d7a6d1444aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:39.397872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:39.401674 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1383/2812] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:47:39.413940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6319fb7-9c70-4369-852d-ad7af6da7d74 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:39.413940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b6319fb7-9c70-4369-852d-ad7af6da7d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:39.482885 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6319fb7-9c70-4369-852d-ad7af6da7d74 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:39.486581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225927da-5768-4ab5-bb74-d5cb27b46eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:39.487190 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225927da-5768-4ab5-bb74-d5cb27b46eec service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:39.487467 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225927da-5768-4ab5-bb74-d5cb27b46eec service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:39.487736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225927da-5768-4ab5-bb74-d5cb27b46eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:39.497992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225927da-5768-4ab5-bb74-d5cb27b46eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:39.498945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225927da-5768-4ab5-bb74-d5cb27b46eec service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:39.510612 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1430/2813] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:39.514829 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77efabc-a470-460d-8db4-d7a6d1444aae None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:39.518092 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b77efabc-a470-460d-8db4-d7a6d1444aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:39.595707 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77efabc-a470-460d-8db4-d7a6d1444aae None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:39.600722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:39.610319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:39.610319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:39.610319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] PATH_INFO: `/v3/projects/3741d7ba9a904c04a079c0a558250003` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:39.610319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3741d7ba9a904c04a079c0a558250003)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:39.610319 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:39.760521 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3741d7ba9a904c04a079c0a558250003', 'request_id': 'req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:39.767438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3741d7ba9a904c04a079c0a558250003', 'request_id': 'req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:39.963752 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3741d7ba9a904c04a079c0a558250003', 'request_id': 'req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:39.976069 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] The token cache is being invalidate because project 3741d7ba9a904c04a079c0a558250003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:47:39.976301 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:39.979571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:39.979571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1384/2814] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:47:39 2022] DELETE /identity/v3/projects/3741d7ba9a904c04a079c0a558250003 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:39.983681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225927da-5768-4ab5-bb74-d5cb27b46eec service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:39.995708 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225927da-5768-4ab5-bb74-d5cb27b46eec service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:40.050559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:40.056416 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:40.060127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:40.060127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:40.060127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] PATH_INFO: `/v3/users/4535aaa5a90540f083d5a1365aed0f58` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:40.060127 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4535aaa5a90540f083d5a1365aed0f58)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:40.061369 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:40.347124 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4535aaa5a90540f083d5a1365aed0f58'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:40.489824 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4535aaa5a90540f083d5a1365aed0f58'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:40.502869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4535aaa5a90540f083d5a1365aed0f58'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:40.503579 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4535aaa5a90540f083d5a1365aed0f58'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:40.510446 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4535aaa5a90540f083d5a1365aed0f58'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:40.516276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1431/2815] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:47:39 2022] DELETE /identity/v3/users/4535aaa5a90540f083d5a1365aed0f58 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:41.080265 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:41.086333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:41.096076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acc0dd-0a02-49a6-abb0-1e0ccb3da390 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:41.159493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:41.160128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:41.160435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:41.160728 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] PATH_INFO: `/v3/projects/2d251076e7b4484d9f3500589b16d7df` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:41.161971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d251076e7b4484d9f3500589b16d7df)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:41.163857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:41.289552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:41.305170 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d251076e7b4484d9f3500589b16d7df', 'request_id': 'req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.312045 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d251076e7b4484d9f3500589b16d7df', 'request_id': 'req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.431953 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d251076e7b4484d9f3500589b16d7df', 'request_id': 'req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.450466 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] The token cache is being invalidate because project 2d251076e7b4484d9f3500589b16d7df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:47:41.450759 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.451408 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.452926 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1385/2816] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:47:41 2022] DELETE /identity/v3/projects/2d251076e7b4484d9f3500589b16d7df => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:47:41.456913 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:41.458484 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:41.472061 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ea4b-d260-414f-b57e-14d8aa6afe08 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:41.542724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:41.543882 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:41.544155 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:41.544424 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] PATH_INFO: `/v3/users/2f0fb264bd7a4901a32ed90933b3d2aa` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:41.545609 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0fb264bd7a4901a32ed90933b3d2aa)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:41.547516 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:41.818165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0fb264bd7a4901a32ed90933b3d2aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.943705 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0fb264bd7a4901a32ed90933b3d2aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.976871 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0fb264bd7a4901a32ed90933b3d2aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.977632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0fb264bd7a4901a32ed90933b3d2aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.985867 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:41.988198 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-1ca15919-0439-4679-931e-402bd4faeea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0fb264bd7a4901a32ed90933b3d2aa'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:41.994331 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1432/2817] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:47:41 2022] DELETE /identity/v3/users/2f0fb264bd7a4901a32ed90933b3d2aa => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:42.545044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:42.545044 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:42.556187 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ebc75-e1bb-4157-82bc-a7acd9df0a60 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:42.620532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:42.621722 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:42.622001 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:42.622272 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] PATH_INFO: `/v3/projects/8936d4d0ab9549628e7819c69211b56f` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:42.623393 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8936d4d0ab9549628e7819c69211b56f)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:42.625366 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:42.778469 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8936d4d0ab9549628e7819c69211b56f', 'request_id': 'req-314cb0a8-7ce3-4a4f-acc4-e19341e76658'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:42.783855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8936d4d0ab9549628e7819c69211b56f', 'request_id': 'req-314cb0a8-7ce3-4a4f-acc4-e19341e76658'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:42.935287 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8936d4d0ab9549628e7819c69211b56f', 'request_id': 'req-314cb0a8-7ce3-4a4f-acc4-e19341e76658'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:42.946921 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] The token cache is being invalidate because project 8936d4d0ab9549628e7819c69211b56f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:47:42.947204 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:42.947822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-314cb0a8-7ce3-4a4f-acc4-e19341e76658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:47:42.949497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1386/2818] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:47:42 2022] DELETE /identity/v3/projects/8936d4d0ab9549628e7819c69211b56f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:47:42.954436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49e02-8798-4609-b413-d9797e012509 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:42.954922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49e02-8798-4609-b413-d9797e012509 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:42.955121 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49e02-8798-4609-b413-d9797e012509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:42.976177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-57a49e02-8798-4609-b413-d9797e012509 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:42.977658 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-57a49e02-8798-4609-b413-d9797e012509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:43.048105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1433/2819] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:43.055876 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac12ad-82c2-460e-8d28-3b0aa2992783 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:43.056161 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac12ad-82c2-460e-8d28-3b0aa2992783 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:43.056435 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac12ad-82c2-460e-8d28-3b0aa2992783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:43.077401 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-86ac12ad-82c2-460e-8d28-3b0aa2992783 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:43.078667 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-86ac12ad-82c2-460e-8d28-3b0aa2992783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:43.086691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1387/2820] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:43.091729 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49e02-8798-4609-b413-d9797e012509 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:43.102045 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49e02-8798-4609-b413-d9797e012509 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:43.107861 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888b1bb-8afc-4436-b889-330c60596640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:43.108462 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888b1bb-8afc-4436-b889-330c60596640 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:43.108741 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888b1bb-8afc-4436-b889-330c60596640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:43.109053 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888b1bb-8afc-4436-b889-330c60596640 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:43.109698 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888b1bb-8afc-4436-b889-330c60596640 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:47:43.110004 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888b1bb-8afc-4436-b889-330c60596640 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:43.111618 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888b1bb-8afc-4436-b889-330c60596640 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:43.124857 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1434/2821] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:47:43.129422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac12ad-82c2-460e-8d28-3b0aa2992783 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:43.139095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac12ad-82c2-460e-8d28-3b0aa2992783 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:43.143836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffab3e-638a-4010-9f9e-e2333dddc938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:43.144434 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffab3e-638a-4010-9f9e-e2333dddc938 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:43.144717 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffab3e-638a-4010-9f9e-e2333dddc938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:43.145007 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffab3e-638a-4010-9f9e-e2333dddc938 admin admin] PATH_INFO: `/v3/projects` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:43.145653 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffab3e-638a-4010-9f9e-e2333dddc938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:43.147536 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffab3e-638a-4010-9f9e-e2333dddc938 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:43.304461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1388/2822] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:47:43 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:47:43.308830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888b1bb-8afc-4436-b889-330c60596640 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:43.319522 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888b1bb-8afc-4436-b889-330c60596640 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:43.379457 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce497b9-a8cb-4d8b-9cbd-f0a7062be242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:43.380068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce497b9-a8cb-4d8b-9cbd-f0a7062be242 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:43.380346 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce497b9-a8cb-4d8b-9cbd-f0a7062be242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:43.380616 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce497b9-a8cb-4d8b-9cbd-f0a7062be242 admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:43.381260 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce497b9-a8cb-4d8b-9cbd-f0a7062be242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:43.383192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce497b9-a8cb-4d8b-9cbd-f0a7062be242 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:43.538195 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1435/2823] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:47:43 2022] POST /identity/v3/users => generated 287 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:47:43.542237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffab3e-638a-4010-9f9e-e2333dddc938 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:43.553584 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffab3e-638a-4010-9f9e-e2333dddc938 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:43.558309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feebb3d-13be-4eb5-a94b-bee87fc0e60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:43.558958 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feebb3d-13be-4eb5-a94b-bee87fc0e60c admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:43.559232 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feebb3d-13be-4eb5-a94b-bee87fc0e60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:43.559497 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feebb3d-13be-4eb5-a94b-bee87fc0e60c admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:43.560128 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feebb3d-13be-4eb5-a94b-bee87fc0e60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:43.561734 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feebb3d-13be-4eb5-a94b-bee87fc0e60c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:43.571779 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1389/2824] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:47:43.575899 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce497b9-a8cb-4d8b-9cbd-f0a7062be242 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:43.585901 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce497b9-a8cb-4d8b-9cbd-f0a7062be242 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:43.598775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0546af-35ec-4236-bc29-6105bef40cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:43.598775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0546af-35ec-4236-bc29-6105bef40cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:43.598775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0546af-35ec-4236-bc29-6105bef40cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:43.598775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0546af-35ec-4236-bc29-6105bef40cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:43.599192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0546af-35ec-4236-bc29-6105bef40cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:43.599192 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0546af-35ec-4236-bc29-6105bef40cbe admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:43.604963 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1436/2825] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:47:43.609844 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feebb3d-13be-4eb5-a94b-bee87fc0e60c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:43.619579 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feebb3d-13be-4eb5-a94b-bee87fc0e60c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:43.624346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388efbf-3448-4464-b88d-53a7e60aa799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:43.625174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388efbf-3448-4464-b88d-53a7e60aa799 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:43.625443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388efbf-3448-4464-b88d-53a7e60aa799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:43.625893 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388efbf-3448-4464-b88d-53a7e60aa799 admin admin] PATH_INFO: `/v3/projects/1d19d73eda154313be8539147f42ec11/users/3afc9f4d5bc3473c837926583d334d31/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:43.635279 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:43.643125 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388efbf-3448-4464-b88d-53a7e60aa799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d19d73eda154313be8539147f42ec11, user_id=3afc9f4d5bc3473c837926583d334d31, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:43.645828 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388efbf-3448-4464-b88d-53a7e60aa799 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:43.796243 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1390/2826] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:47:43 2022] PUT /identity/v3/projects/1d19d73eda154313be8539147f42ec11/users/3afc9f4d5bc3473c837926583d334d31/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:47:43.801366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de77a5-9a83-4fe2-b967-3d8c34b25b2f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:43.801649 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de77a5-9a83-4fe2-b967-3d8c34b25b2f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:43.801981 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de77a5-9a83-4fe2-b967-3d8c34b25b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:43.821088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-02de77a5-9a83-4fe2-b967-3d8c34b25b2f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:43.822267 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-02de77a5-9a83-4fe2-b967-3d8c34b25b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:43.883394 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1437/2827] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:43.894063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388efbf-3448-4464-b88d-53a7e60aa799 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:43.895888 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0388efbf-3448-4464-b88d-53a7e60aa799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:43.958096 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388efbf-3448-4464-b88d-53a7e60aa799 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:43.962483 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e1124-7ea5-4b37-9c13-e0435b60af93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:43.963129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e1124-7ea5-4b37-9c13-e0435b60af93 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:43.963398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e1124-7ea5-4b37-9c13-e0435b60af93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:43.963765 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e1124-7ea5-4b37-9c13-e0435b60af93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:43.975587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e1124-7ea5-4b37-9c13-e0435b60af93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:43.976698 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e1124-7ea5-4b37-9c13-e0435b60af93 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:43.987221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1391/2828] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:45.668437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b58c7-9a71-467d-8974-868c0be09d8f None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:45.668437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b58c7-9a71-467d-8974-868c0be09d8f None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:45.668437 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7b58c7-9a71-467d-8974-868c0be09d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:45.672564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:47:45.695357 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-9d7b58c7-9a71-467d-8974-868c0be09d8f None None] MFA Rules not processed for user `3afc9f4d5bc3473c837926583d334d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:45.696622 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-9d7b58c7-9a71-467d-8974-868c0be09d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:45.764736 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1438/2829] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:47:45.776686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e1124-7ea5-4b37-9c13-e0435b60af93 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:45.778224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-377e1124-7ea5-4b37-9c13-e0435b60af93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:45.835856 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e1124-7ea5-4b37-9c13-e0435b60af93 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:45.839428 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5298624-f85f-41dc-a751-876365364f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:45.840073 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5298624-f85f-41dc-a751-876365364f3c service cinder] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:45.840345 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5298624-f85f-41dc-a751-876365364f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:45.840609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5298624-f85f-41dc-a751-876365364f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:45.849571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5298624-f85f-41dc-a751-876365364f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:45.851050 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5298624-f85f-41dc-a751-876365364f3c service cinder] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:45.861347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1392/2830] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:49.981542 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b58c7-9a71-467d-8974-868c0be09d8f None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:49.992133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7b58c7-9a71-467d-8974-868c0be09d8f None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:49.995912 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:49.997116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:49.997406 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:49.997680 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:49.999477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:50.085082 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:50.086539 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:50.106024 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1439/2831] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:51.874317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5298624-f85f-41dc-a751-876365364f3c service cinder] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:51.884126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5298624-f85f-41dc-a751-876365364f3c service cinder] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:51.888454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a2762-6f4c-49af-8a0e-a6a82f15b210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:51.889160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a2762-6f4c-49af-8a0e-a6a82f15b210 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:51.889482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a2762-6f4c-49af-8a0e-a6a82f15b210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:51.889622 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a2762-6f4c-49af-8a0e-a6a82f15b210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:51.891575 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2d3a2762-6f4c-49af-8a0e-a6a82f15b210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:51.965598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a2762-6f4c-49af-8a0e-a6a82f15b210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:51.967075 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a2762-6f4c-49af-8a0e-a6a82f15b210 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:51.980572 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1393/2832] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:53.179577 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:53.181319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:53.193388 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a5e88-ce36-476d-9b2b-1281d52af4a1 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:53.196982 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80000b0-03fa-40f5-8344-5664d14c26ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:53.197583 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80000b0-03fa-40f5-8344-5664d14c26ea service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:53.197854 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80000b0-03fa-40f5-8344-5664d14c26ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:53.198122 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80000b0-03fa-40f5-8344-5664d14c26ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:53.201102 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e80000b0-03fa-40f5-8344-5664d14c26ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:53.215088 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80000b0-03fa-40f5-8344-5664d14c26ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:53.216511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80000b0-03fa-40f5-8344-5664d14c26ea service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:53.228193 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1440/2833] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:54.648454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a2762-6f4c-49af-8a0e-a6a82f15b210 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:47:54.659635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a2762-6f4c-49af-8a0e-a6a82f15b210 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:47:54.664787 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:47:54.665642 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:54.665980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:54.666386 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:54.669091 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:54.680030 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:47:54.681530 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:47:54.690908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1394/2834] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:47:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:47:57.322643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f81b-2d6e-436d-bba9-2a56265b5121 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:47:57.323287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f81b-2d6e-436d-bba9-2a56265b5121 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:47:57.323287 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f81b-2d6e-436d-bba9-2a56265b5121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:47:57.350144 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-e854f81b-2d6e-436d-bba9-2a56265b5121 None None] MFA Rules not processed for user `3afc9f4d5bc3473c837926583d334d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:47:57.351906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e854f81b-2d6e-436d-bba9-2a56265b5121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:47:57.360300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1441/2835] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:47:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:04.686639 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:07.342453 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=81 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:25.513529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:25.515068 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:25.533431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2adcb4-bcb0-4196-a51d-4f6d459b426a service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:25.539131 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:25.539757 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:25.540031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:25.540297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:25.542228 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:25.556142 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:25.557782 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:25.570552 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1395/2836] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:48:26.646526 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f81b-2d6e-436d-bba9-2a56265b5121 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:26.648078 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-e854f81b-2d6e-436d-bba9-2a56265b5121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:26.669177 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f81b-2d6e-436d-bba9-2a56265b5121 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:26.675250 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4875468-b331-41f8-8279-0ab977e63c8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:26.675897 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875468-b331-41f8-8279-0ab977e63c8e service nova] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:26.676165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875468-b331-41f8-8279-0ab977e63c8e service nova] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:26.676439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875468-b331-41f8-8279-0ab977e63c8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:26.678274 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d4875468-b331-41f8-8279-0ab977e63c8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:26.693571 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4875468-b331-41f8-8279-0ab977e63c8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:26.695238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4875468-b331-41f8-8279-0ab977e63c8e service nova] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:26.707561 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1442/2837] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:48:31.678268 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:31.679694 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:31.694214 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dd5d2-01d7-494c-9e3a-17c8ffff6963 service nova] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:31.699014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:31.700720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:31.700720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:31.700720 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] PATH_INFO: `/v3/users/7ac44c0375c04784b1fc10805b64a35f` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:31.703260 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac44c0375c04784b1fc10805b64a35f)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:31.705418 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:31.942563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac44c0375c04784b1fc10805b64a35f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:31.943754 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac44c0375c04784b1fc10805b64a35f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:31.944908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac44c0375c04784b1fc10805b64a35f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:31.957712 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac44c0375c04784b1fc10805b64a35f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:31.974226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac44c0375c04784b1fc10805b64a35f'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:32.075443 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1396/2838] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:48:31 2022] DELETE /identity/v3/users/7ac44c0375c04784b1fc10805b64a35f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:32.081327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124e4d0-2dbb-4771-aa95-fc7d0d5598b5 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:32.081890 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124e4d0-2dbb-4771-aa95-fc7d0d5598b5 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:32.082105 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124e4d0-2dbb-4771-aa95-fc7d0d5598b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:32.110497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d124e4d0-2dbb-4771-aa95-fc7d0d5598b5 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:32.111734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d124e4d0-2dbb-4771-aa95-fc7d0d5598b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:32.190874 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1443/2839] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:32.203244 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:32.205195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:32.277905 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a75b3-461f-4e94-b740-9a7e534b757d admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:32.282672 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d78cf-cf69-4267-86b2-42d9ab039f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:32.283297 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d78cf-cf69-4267-86b2-42d9ab039f90 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:32.283571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d78cf-cf69-4267-86b2-42d9ab039f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:32.283840 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d78cf-cf69-4267-86b2-42d9ab039f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:32.294398 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d78cf-cf69-4267-86b2-42d9ab039f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:32.297075 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d78cf-cf69-4267-86b2-42d9ab039f90 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:32.308035 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1397/2840] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:48:32.842597 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124e4d0-2dbb-4771-aa95-fc7d0d5598b5 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:32.844168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d124e4d0-2dbb-4771-aa95-fc7d0d5598b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:32.859802 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124e4d0-2dbb-4771-aa95-fc7d0d5598b5 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:32.866419 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:32.867093 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:32.867417 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:32.870288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] PATH_INFO: `/v3/projects/72bc32a0f8ad4a1fb4feb90edb84b3c0` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:32.870288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72bc32a0f8ad4a1fb4feb90edb84b3c0)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:32.871697 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:33.055514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72bc32a0f8ad4a1fb4feb90edb84b3c0', 'request_id': 'req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:33.063206 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72bc32a0f8ad4a1fb4feb90edb84b3c0', 'request_id': 'req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:33.148943 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72bc32a0f8ad4a1fb4feb90edb84b3c0', 'request_id': 'req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:33.167841 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] The token cache is being invalidate because project 72bc32a0f8ad4a1fb4feb90edb84b3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:48:33.168289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:33.169194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-7ddd302d-6ff8-4480-babd-67a8f2bd46e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:33.171608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1444/2841] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:48:32 2022] DELETE /identity/v3/projects/72bc32a0f8ad4a1fb4feb90edb84b3c0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:33.178059 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7509d4-f855-463e-8af1-359fb6476d5f None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:33.180822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7509d4-f855-463e-8af1-359fb6476d5f None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:33.180822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7509d4-f855-463e-8af1-359fb6476d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:33.211580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-db7509d4-f855-463e-8af1-359fb6476d5f None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:33.213036 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-db7509d4-f855-463e-8af1-359fb6476d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:33.292708 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1398/2842] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:33.301373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a776b-267c-4832-9d8b-ddf0b1f02578 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:33.301707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a776b-267c-4832-9d8b-ddf0b1f02578 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:33.302049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a776b-267c-4832-9d8b-ddf0b1f02578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:33.323766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-289a776b-267c-4832-9d8b-ddf0b1f02578 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:33.325039 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-289a776b-267c-4832-9d8b-ddf0b1f02578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:33.334745 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1445/2843] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:33.339115 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7509d4-f855-463e-8af1-359fb6476d5f None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:33.346023 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:33.351731 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7509d4-f855-463e-8af1-359fb6476d5f None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:33.356487 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d921c62-28cd-436f-b0d7-d283b414a9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:33.357166 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d921c62-28cd-436f-b0d7-d283b414a9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:33.360496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d921c62-28cd-436f-b0d7-d283b414a9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:33.360496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d921c62-28cd-436f-b0d7-d283b414a9c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:33.360496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d921c62-28cd-436f-b0d7-d283b414a9c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81707) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:48:33.360496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d921c62-28cd-436f-b0d7-d283b414a9c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:33.360496 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d921c62-28cd-436f-b0d7-d283b414a9c4 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:33.373573 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1399/2844] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:48:33.379101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a776b-267c-4832-9d8b-ddf0b1f02578 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:33.393471 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a776b-267c-4832-9d8b-ddf0b1f02578 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:33.398620 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec2527-1fae-4fd9-af29-d856c69b6f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:33.399240 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec2527-1fae-4fd9-af29-d856c69b6f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:33.399518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec2527-1fae-4fd9-af29-d856c69b6f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:33.399786 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec2527-1fae-4fd9-af29-d856c69b6f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:33.400422 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec2527-1fae-4fd9-af29-d856c69b6f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:33.402452 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec2527-1fae-4fd9-af29-d856c69b6f60 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:33.524791 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1446/2845] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:48:33 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:48:33.529117 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d921c62-28cd-436f-b0d7-d283b414a9c4 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:33.539959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d921c62-28cd-436f-b0d7-d283b414a9c4 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:33.606623 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5908d-f105-4df2-b357-c7b89a5c5e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:33.607255 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5908d-f105-4df2-b357-c7b89a5c5e2c admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:33.608226 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5908d-f105-4df2-b357-c7b89a5c5e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:33.608647 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5908d-f105-4df2-b357-c7b89a5c5e2c admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:33.609392 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5908d-f105-4df2-b357-c7b89a5c5e2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:33.611342 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5908d-f105-4df2-b357-c7b89a5c5e2c admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:33.710823 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1400/2846] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:48:33 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:48:33.713288 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec2527-1fae-4fd9-af29-d856c69b6f60 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:33.729711 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec2527-1fae-4fd9-af29-d856c69b6f60 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:33.749080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df5fc1-8140-4c7a-8f7a-c29f9f6b8067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:33.749080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df5fc1-8140-4c7a-8f7a-c29f9f6b8067 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:33.749080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df5fc1-8140-4c7a-8f7a-c29f9f6b8067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:33.749080 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df5fc1-8140-4c7a-8f7a-c29f9f6b8067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:33.749505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df5fc1-8140-4c7a-8f7a-c29f9f6b8067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:33.749505 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df5fc1-8140-4c7a-8f7a-c29f9f6b8067 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:33.756104 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1447/2847] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:48:33.757250 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5908d-f105-4df2-b357-c7b89a5c5e2c admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:33.769300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5908d-f105-4df2-b357-c7b89a5c5e2c admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:33.774987 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e20fb-312c-4910-ad92-acdc98fa6297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:33.775596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e20fb-312c-4910-ad92-acdc98fa6297 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:33.776038 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e20fb-312c-4910-ad92-acdc98fa6297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:33.776275 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e20fb-312c-4910-ad92-acdc98fa6297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:33.776951 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e20fb-312c-4910-ad92-acdc98fa6297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:33.778548 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e20fb-312c-4910-ad92-acdc98fa6297 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:33.791041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1401/2848] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 11:48:33.795217 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df5fc1-8140-4c7a-8f7a-c29f9f6b8067 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:33.806880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df5fc1-8140-4c7a-8f7a-c29f9f6b8067 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:33.812321 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d3b96-6ce8-4efd-833e-3978cbb3639a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:33.813101 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d3b96-6ce8-4efd-833e-3978cbb3639a admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:33.813454 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d3b96-6ce8-4efd-833e-3978cbb3639a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:33.813738 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d3b96-6ce8-4efd-833e-3978cbb3639a admin admin] PATH_INFO: `/v3/projects/9a70cdccbcb84216ae61c72d6736bd63/users/5591708bfed04ac185d4c610e8b76125/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:33.829852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d3b96-6ce8-4efd-833e-3978cbb3639a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a70cdccbcb84216ae61c72d6736bd63, user_id=5591708bfed04ac185d4c610e8b76125, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:33.832682 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d3b96-6ce8-4efd-833e-3978cbb3639a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:33.901995 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1448/2849] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:48:33 2022] PUT /identity/v3/projects/9a70cdccbcb84216ae61c72d6736bd63/users/5591708bfed04ac185d4c610e8b76125/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:33.907929 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b60b-b7aa-4894-b1d7-b8bc327a2bb6 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:33.908280 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b60b-b7aa-4894-b1d7-b8bc327a2bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:33.908626 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b60b-b7aa-4894-b1d7-b8bc327a2bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:33.927261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-e250b60b-b7aa-4894-b1d7-b8bc327a2bb6 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:33.928605 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e250b60b-b7aa-4894-b1d7-b8bc327a2bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:33.996730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1402/2850] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:34.008138 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d3b96-6ce8-4efd-833e-3978cbb3639a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:34.010014 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-402d3b96-6ce8-4efd-833e-3978cbb3639a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:34.021706 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:34.078319 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d3b96-6ce8-4efd-833e-3978cbb3639a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:34.082233 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26e021-8570-4e08-aa50-4d4d7cd1ed3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:34.082856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26e021-8570-4e08-aa50-4d4d7cd1ed3d service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:34.083132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26e021-8570-4e08-aa50-4d4d7cd1ed3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:34.083402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26e021-8570-4e08-aa50-4d4d7cd1ed3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:34.118938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26e021-8570-4e08-aa50-4d4d7cd1ed3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:34.120397 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26e021-8570-4e08-aa50-4d4d7cd1ed3d service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:34.135091 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1449/2851] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:48:34.517587 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b60b-b7aa-4894-b1d7-b8bc327a2bb6 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:34.527074 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b60b-b7aa-4894-b1d7-b8bc327a2bb6 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:34.531304 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57dce6-ffff-4775-8a4d-cb2d6cb51bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:34.531895 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57dce6-ffff-4775-8a4d-cb2d6cb51bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:34.532174 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57dce6-ffff-4775-8a4d-cb2d6cb51bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:34.532482 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57dce6-ffff-4775-8a4d-cb2d6cb51bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:34.534220 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8e57dce6-ffff-4775-8a4d-cb2d6cb51bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:34.547917 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57dce6-ffff-4775-8a4d-cb2d6cb51bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:34.549371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57dce6-ffff-4775-8a4d-cb2d6cb51bc6 service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:34.560508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1403/2852] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:48:35.350009 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076c925-e1c2-4a5a-876c-4b92e026c143 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:35.350511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076c925-e1c2-4a5a-876c-4b92e026c143 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:35.350636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076c925-e1c2-4a5a-876c-4b92e026c143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:35.372132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-d076c925-e1c2-4a5a-876c-4b92e026c143 None None] MFA Rules not processed for user `3afc9f4d5bc3473c837926583d334d31`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:35.374139 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d076c925-e1c2-4a5a-876c-4b92e026c143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:35.436904 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1450/2853] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:35.908825 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57dce6-ffff-4775-8a4d-cb2d6cb51bc6 service neutron] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:35.918300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57dce6-ffff-4775-8a4d-cb2d6cb51bc6 service neutron] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:35.923251 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1ee6e-0740-4729-a75a-8ab5338cba0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:35.924067 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1ee6e-0740-4729-a75a-8ab5338cba0f service neutron] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:35.924412 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1ee6e-0740-4729-a75a-8ab5338cba0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:35.924690 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1ee6e-0740-4729-a75a-8ab5338cba0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:35.926635 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-43e1ee6e-0740-4729-a75a-8ab5338cba0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:35.937490 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1ee6e-0740-4729-a75a-8ab5338cba0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:35.939413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1ee6e-0740-4729-a75a-8ab5338cba0f service neutron] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:35.948061 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1404/2854] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:48:36.584785 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076c925-e1c2-4a5a-876c-4b92e026c143 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:36.593149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d076c925-e1c2-4a5a-876c-4b92e026c143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:36.692734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076c925-e1c2-4a5a-876c-4b92e026c143 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:36.700199 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0a92a-7b0c-4448-b143-c528c38532d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:36.700838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0a92a-7b0c-4448-b143-c528c38532d9 service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:36.701740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0a92a-7b0c-4448-b143-c528c38532d9 service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:36.702018 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0a92a-7b0c-4448-b143-c528c38532d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:36.703940 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-47a0a92a-7b0c-4448-b143-c528c38532d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:36.716321 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0a92a-7b0c-4448-b143-c528c38532d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:36.718244 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0a92a-7b0c-4448-b143-c528c38532d9 service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:36.728957 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1451/2855] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:48:43.777499 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7544f6-d10c-41eb-8e69-b07e63d01521 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:43.778697 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7544f6-d10c-41eb-8e69-b07e63d01521 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:43.779300 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7544f6-d10c-41eb-8e69-b07e63d01521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:43.842934 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-8d7544f6-d10c-41eb-8e69-b07e63d01521 None None] MFA Rules not processed for user `5591708bfed04ac185d4c610e8b76125`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:43.844172 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-8d7544f6-d10c-41eb-8e69-b07e63d01521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:43.895305 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:43.947586 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1405/2856] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:43.948345 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0a92a-7b0c-4448-b143-c528c38532d9 service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:43.965798 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0a92a-7b0c-4448-b143-c528c38532d9 service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:43.975602 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c11013-9b04-4b30-825b-ee20248868e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:43.976278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c11013-9b04-4b30-825b-ee20248868e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:43.976555 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c11013-9b04-4b30-825b-ee20248868e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:43.976847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c11013-9b04-4b30-825b-ee20248868e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:43.977496 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c11013-9b04-4b30-825b-ee20248868e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:43.984025 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c11013-9b04-4b30-825b-ee20248868e4 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:44.149979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1452/2857] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:48:43 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 11:48:44.154805 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7544f6-d10c-41eb-8e69-b07e63d01521 None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:44.167159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7544f6-d10c-41eb-8e69-b07e63d01521 None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:44.256271 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6de858-7f72-4bb6-8db5-6541b13e4144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:44.257843 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6de858-7f72-4bb6-8db5-6541b13e4144 admin admin] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:44.258544 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6de858-7f72-4bb6-8db5-6541b13e4144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:44.258835 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6de858-7f72-4bb6-8db5-6541b13e4144 admin admin] PATH_INFO: `/v3/users` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:44.259477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6de858-7f72-4bb6-8db5-6541b13e4144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:44.265912 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6de858-7f72-4bb6-8db5-6541b13e4144 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:44.411956 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c11013-9b04-4b30-825b-ee20248868e4 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:44.419076 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1406/2858] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:48:44 2022] POST /identity/v3/users => generated 299 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:48:44.426050 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c11013-9b04-4b30-825b-ee20248868e4 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:44.431257 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376ab6d-9396-4bb8-a995-353f104ea02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:44.434238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376ab6d-9396-4bb8-a995-353f104ea02c admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:44.434238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376ab6d-9396-4bb8-a995-353f104ea02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:44.434238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376ab6d-9396-4bb8-a995-353f104ea02c admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:44.434238 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376ab6d-9396-4bb8-a995-353f104ea02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:44.436514 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376ab6d-9396-4bb8-a995-353f104ea02c admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:44.453942 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1453/2859] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:48:44.458419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6de858-7f72-4bb6-8db5-6541b13e4144 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:44.480988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6de858-7f72-4bb6-8db5-6541b13e4144 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:44.491804 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6d0e9-3edc-41fd-8005-9ca1865d4267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:44.492949 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6d0e9-3edc-41fd-8005-9ca1865d4267 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:44.493476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6d0e9-3edc-41fd-8005-9ca1865d4267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:44.494003 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6d0e9-3edc-41fd-8005-9ca1865d4267 admin admin] PATH_INFO: `/v3/projects/e53dcc406e794017a18e3f7b66f5a5cd/users/14a74be492a5403c8c9750bb6d6c4c33/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:44.521391 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6d0e9-3edc-41fd-8005-9ca1865d4267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53dcc406e794017a18e3f7b66f5a5cd, user_id=14a74be492a5403c8c9750bb6d6c4c33, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:44.524032 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6d0e9-3edc-41fd-8005-9ca1865d4267 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:44.645477 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1407/2860] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:48:44 2022] PUT /identity/v3/projects/e53dcc406e794017a18e3f7b66f5a5cd/users/14a74be492a5403c8c9750bb6d6c4c33/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:44.648515 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376ab6d-9396-4bb8-a995-353f104ea02c admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:44.662536 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376ab6d-9396-4bb8-a995-353f104ea02c admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:44.686197 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:44.785770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d5891-52d8-4930-a0ca-e39d8dd1f953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:44.786762 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d5891-52d8-4930-a0ca-e39d8dd1f953 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:44.788123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d5891-52d8-4930-a0ca-e39d8dd1f953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:44.789366 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d5891-52d8-4930-a0ca-e39d8dd1f953 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:44.793407 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d5891-52d8-4930-a0ca-e39d8dd1f953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:44.799473 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d5891-52d8-4930-a0ca-e39d8dd1f953 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:44.824477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1454/2861] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:48:44.829628 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6d0e9-3edc-41fd-8005-9ca1865d4267 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:44.853523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6d0e9-3edc-41fd-8005-9ca1865d4267 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:44.866696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cef56-5a62-42c6-a658-7766e81d6501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:44.868930 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cef56-5a62-42c6-a658-7766e81d6501 admin admin] REQUEST_METHOD: `PUT` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:44.870349 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cef56-5a62-42c6-a658-7766e81d6501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:44.870723 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cef56-5a62-42c6-a658-7766e81d6501 admin admin] PATH_INFO: `/v3/projects/e53dcc406e794017a18e3f7b66f5a5cd/users/14a74be492a5403c8c9750bb6d6c4c33/roles/e9c4afeedebc4a3db8d3713e9671a05c` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:44.874816 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cef56-5a62-42c6-a658-7766e81d6501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53dcc406e794017a18e3f7b66f5a5cd, user_id=14a74be492a5403c8c9750bb6d6c4c33, role_id=e9c4afeedebc4a3db8d3713e9671a05c)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:44.879598 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cef56-5a62-42c6-a658-7766e81d6501 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:44.896553 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1408/2862] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:48:44 2022] PUT /identity/v3/projects/e53dcc406e794017a18e3f7b66f5a5cd/users/14a74be492a5403c8c9750bb6d6c4c33/roles/e9c4afeedebc4a3db8d3713e9671a05c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:46.560058 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d5891-52d8-4930-a0ca-e39d8dd1f953 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:46.562992 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-fe2d5891-52d8-4930-a0ca-e39d8dd1f953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:46.583743 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d5891-52d8-4930-a0ca-e39d8dd1f953 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:46.769207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:46.769945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:46.770228 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:46.770500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] PATH_INFO: `/v3/users/6bbb460842f44f4d81acb507f04469fe` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:46.771728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bbb460842f44f4d81acb507f04469fe)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:46.773779 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:47.001900 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bbb460842f44f4d81acb507f04469fe'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:47.105728 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bbb460842f44f4d81acb507f04469fe'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:47.134175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bbb460842f44f4d81acb507f04469fe'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:47.135312 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bbb460842f44f4d81acb507f04469fe'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:47.156183 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bbb460842f44f4d81acb507f04469fe'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:47.169466 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1455/2863] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:48:46 2022] DELETE /identity/v3/users/6bbb460842f44f4d81acb507f04469fe => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:47.172277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde2bb8-8071-44db-b0a1-9ac884f11eda None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:47.172789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde2bb8-8071-44db-b0a1-9ac884f11eda None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:47.172789 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde2bb8-8071-44db-b0a1-9ac884f11eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:47.217229 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-4fde2bb8-8071-44db-b0a1-9ac884f11eda None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:47.221488 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4fde2bb8-8071-44db-b0a1-9ac884f11eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:47.370824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1409/2864] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:47.387458 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:47.389910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:47.462439 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:47.516227 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114dec-8dc4-44f7-84d1-e0827640300b admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:47.526772 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b725d3-bc99-4535-a0ff-ae9a343c7fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:47.527436 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b725d3-bc99-4535-a0ff-ae9a343c7fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:47.527710 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b725d3-bc99-4535-a0ff-ae9a343c7fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:47.527979 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b725d3-bc99-4535-a0ff-ae9a343c7fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:47.553718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b725d3-bc99-4535-a0ff-ae9a343c7fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:47.555363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b725d3-bc99-4535-a0ff-ae9a343c7fd1 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:47.578599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1456/2865] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:48:47.811873 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde2bb8-8071-44db-b0a1-9ac884f11eda None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:47.813431 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4fde2bb8-8071-44db-b0a1-9ac884f11eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:47.833730 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde2bb8-8071-44db-b0a1-9ac884f11eda None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:47.849486 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:47.850862 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:47.851367 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:47.852095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] PATH_INFO: `/v3/users/0b82b6f568bd41fa9d6b44facb766e0b` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:47.855176 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b82b6f568bd41fa9d6b44facb766e0b)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:47.859990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:47.905198 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:48.150095 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b82b6f568bd41fa9d6b44facb766e0b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.150563 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b82b6f568bd41fa9d6b44facb766e0b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.151559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b82b6f568bd41fa9d6b44facb766e0b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.172954 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b82b6f568bd41fa9d6b44facb766e0b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.198525 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b82b6f568bd41fa9d6b44facb766e0b'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.309239 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1410/2866] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:48:47 2022] DELETE /identity/v3/users/0b82b6f568bd41fa9d6b44facb766e0b => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:48.397408 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b725d3-bc99-4535-a0ff-ae9a343c7fd1 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:48.403537 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-47b725d3-bc99-4535-a0ff-ae9a343c7fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:48.433038 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b725d3-bc99-4535-a0ff-ae9a343c7fd1 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:48.568334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:48.569276 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:48.569605 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:48.570363 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] PATH_INFO: `/v3/projects/c3aca5bfdf474fc7a7ddb0f9fc0d5e2f` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:48.579029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3aca5bfdf474fc7a7ddb0f9fc0d5e2f)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:48.588086 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:48.673347 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:48.674855 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:48.694147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aec6e-996e-4717-923b-481700584d33 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:48.700323 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:48.701492 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:48.701970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:48.702457 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] PATH_INFO: `/v3/projects/c4ef3fc15a0147e39228729d2b5b2631` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:48.704325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ef3fc15a0147e39228729d2b5b2631)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:48.706737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:48.743859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3aca5bfdf474fc7a7ddb0f9fc0d5e2f', 'request_id': 'req-4c446810-681f-489d-95fd-25c5160c19cc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.764373 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3aca5bfdf474fc7a7ddb0f9fc0d5e2f', 'request_id': 'req-4c446810-681f-489d-95fd-25c5160c19cc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.876469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3aca5bfdf474fc7a7ddb0f9fc0d5e2f', 'request_id': 'req-4c446810-681f-489d-95fd-25c5160c19cc'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.876969 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ef3fc15a0147e39228729d2b5b2631', 'request_id': 'req-6795407a-c0c7-4300-a958-4b2d9f928665'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.885686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ef3fc15a0147e39228729d2b5b2631', 'request_id': 'req-6795407a-c0c7-4300-a958-4b2d9f928665'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.894398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] The token cache is being invalidate because project c3aca5bfdf474fc7a7ddb0f9fc0d5e2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:48:48.895123 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.896026 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4c446810-681f-489d-95fd-25c5160c19cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:48.897853 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1457/2867] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:48:48 2022] DELETE /identity/v3/projects/c3aca5bfdf474fc7a7ddb0f9fc0d5e2f => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:48.907846 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16202c-540f-4f74-b0ca-97dfdcfc2f33 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:48.908640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16202c-540f-4f74-b0ca-97dfdcfc2f33 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:48.914318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16202c-540f-4f74-b0ca-97dfdcfc2f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:48.945433 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-df16202c-540f-4f74-b0ca-97dfdcfc2f33 None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:48.947342 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-df16202c-540f-4f74-b0ca-97dfdcfc2f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:48.991473 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ef3fc15a0147e39228729d2b5b2631', 'request_id': 'req-6795407a-c0c7-4300-a958-4b2d9f928665'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:49.016261 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] The token cache is being invalidate because project c4ef3fc15a0147e39228729d2b5b2631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:48:49.016937 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:49.018609 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:49.021002 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1411/2868] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:48:48 2022] DELETE /identity/v3/projects/c4ef3fc15a0147e39228729d2b5b2631 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:49.026378 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:49.027980 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:49.056300 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1458/2869] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:49.068189 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fc8d0-5a54-40b8-9c9a-9cef555fb8de None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:49.068829 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fc8d0-5a54-40b8-9c9a-9cef555fb8de None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:49.069207 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fc8d0-5a54-40b8-9c9a-9cef555fb8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:49.096906 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-b32fc8d0-5a54-40b8-9c9a-9cef555fb8de None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:49.099334 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-b32fc8d0-5a54-40b8-9c9a-9cef555fb8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:49.115914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1459/2870] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:49.121678 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fc8d0-5a54-40b8-9c9a-9cef555fb8de None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:49.135229 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fc8d0-5a54-40b8-9c9a-9cef555fb8de None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:49.136041 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795407a-c0c7-4300-a958-4b2d9f928665 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:49.142401 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be550aa-6c1a-4ea9-81cd-367a6243d294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:49.142870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:49.143323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be550aa-6c1a-4ea9-81cd-367a6243d294 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:49.143945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be550aa-6c1a-4ea9-81cd-367a6243d294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:49.144071 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:49.144425 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:49.144636 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be550aa-6c1a-4ea9-81cd-367a6243d294 admin admin] PATH_INFO: `/v3/domains` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:49.144865 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] PATH_INFO: `/v3/users/6eee4d6ce51d4d2fa054ff69b0b61ea2` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:49.145718 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be550aa-6c1a-4ea9-81cd-367a6243d294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81706) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 11:48:49.146208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be550aa-6c1a-4ea9-81cd-367a6243d294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:49.150492 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be550aa-6c1a-4ea9-81cd-367a6243d294 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:49.167574 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eee4d6ce51d4d2fa054ff69b0b61ea2)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:49.170438 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1460/2871] 10.222.0.84 () {58 vars in 1153 bytes} [Wed Aug 24 11:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 11:48:49.173180 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:49.175236 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be550aa-6c1a-4ea9-81cd-367a6243d294 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:49.188964 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be550aa-6c1a-4ea9-81cd-367a6243d294 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:49.195187 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b613ef-de8d-4712-9742-cffcea3f041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:49.195939 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b613ef-de8d-4712-9742-cffcea3f041a admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:49.196361 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b613ef-de8d-4712-9742-cffcea3f041a admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:49.196734 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b613ef-de8d-4712-9742-cffcea3f041a admin admin] PATH_INFO: `/v3/projects` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:49.197650 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b613ef-de8d-4712-9742-cffcea3f041a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:49.199813 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b613ef-de8d-4712-9742-cffcea3f041a admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:49.339258 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1461/2872] 10.222.0.84 () {60 vars in 1153 bytes} [Wed Aug 24 11:48:49 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:48:49.345432 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b613ef-de8d-4712-9742-cffcea3f041a admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:49.364619 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eee4d6ce51d4d2fa054ff69b0b61ea2'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:49.365615 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eee4d6ce51d4d2fa054ff69b0b61ea2'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:49.366550 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eee4d6ce51d4d2fa054ff69b0b61ea2'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:49.376753 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eee4d6ce51d4d2fa054ff69b0b61ea2'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:49.396974 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eee4d6ce51d4d2fa054ff69b0b61ea2'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:49.435184 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b613ef-de8d-4712-9742-cffcea3f041a admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:49.441324 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52d36c-4888-46b8-a360-85ca915454fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:49.442328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52d36c-4888-46b8-a360-85ca915454fe admin admin] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:49.442852 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52d36c-4888-46b8-a360-85ca915454fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:49.445245 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:49.446168 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52d36c-4888-46b8-a360-85ca915454fe admin admin] PATH_INFO: `/v3/users` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:49.447167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52d36c-4888-46b8-a360-85ca915454fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:49.450303 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52d36c-4888-46b8-a360-85ca915454fe admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:49.484924 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1412/2873] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:48:49 2022] DELETE /identity/v3/users/6eee4d6ce51d4d2fa054ff69b0b61ea2 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:48:49.589754 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1462/2874] 10.222.0.84 () {60 vars in 1144 bytes} [Wed Aug 24 11:48:49 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 11:48:49.596014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:49.600215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:49.742737 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc64568-a64b-4263-b2c6-603a072b73f7 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:49.745208 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a0150-4db4-40f0-b76d-8079fb438644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:49.745836 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a0150-4db4-40f0-b76d-8079fb438644 admin admin] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:49.746112 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a0150-4db4-40f0-b76d-8079fb438644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:49.746382 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a0150-4db4-40f0-b76d-8079fb438644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:49.747031 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a0150-4db4-40f0-b76d-8079fb438644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:49.749559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a0150-4db4-40f0-b76d-8079fb438644 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:49.762795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1413/2875] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:48:49.766929 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52d36c-4888-46b8-a360-85ca915454fe admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:49.788179 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52d36c-4888-46b8-a360-85ca915454fe admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:49.804655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af018ca-fe4e-45e2-ba3c-f5bf50092590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:49.804655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af018ca-fe4e-45e2-ba3c-f5bf50092590 admin admin] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:49.804655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af018ca-fe4e-45e2-ba3c-f5bf50092590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:49.804655 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af018ca-fe4e-45e2-ba3c-f5bf50092590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:49.815914 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af018ca-fe4e-45e2-ba3c-f5bf50092590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:49.817740 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af018ca-fe4e-45e2-ba3c-f5bf50092590 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:49.833182 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a0150-4db4-40f0-b76d-8079fb438644 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:49.834940 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2c2a0150-4db4-40f0-b76d-8079fb438644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:49.836975 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1463/2876] 10.222.0.84 () {58 vars in 1122 bytes} [Wed Aug 24 11:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 11:48:49.840737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af018ca-fe4e-45e2-ba3c-f5bf50092590 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:49.851377 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af018ca-fe4e-45e2-ba3c-f5bf50092590 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:49.859465 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3c3c-9148-4b9e-a783-6e80374fe48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:49.860273 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3c3c-9148-4b9e-a783-6e80374fe48b admin admin] REQUEST_METHOD: `PUT` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:49.860651 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3c3c-9148-4b9e-a783-6e80374fe48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:49.861068 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3c3c-9148-4b9e-a783-6e80374fe48b admin admin] PATH_INFO: `/v3/projects/71a2d9f0a96b4c8b82f343ecc8cc26ba/users/c7974c9be50c4a05b8d067fa0e4d0a96/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:49.861686 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a0150-4db4-40f0-b76d-8079fb438644 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:49.880537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:49.881896 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3c3c-9148-4b9e-a783-6e80374fe48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a2d9f0a96b4c8b82f343ecc8cc26ba, user_id=c7974c9be50c4a05b8d067fa0e4d0a96, role_id=bbd3afeaf4a9469aa9d6cbf7a5cb9f18)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:49.882116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:49.882556 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:49.882990 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] PATH_INFO: `/v3/projects/7609f88d3e834d29a09d17d3a0ba8614` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:49.889684 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3c3c-9148-4b9e-a783-6e80374fe48b admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:49.899459 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7609f88d3e834d29a09d17d3a0ba8614)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:49.901581 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:49.989988 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1464/2877] 10.222.0.84 () {60 vars in 1483 bytes} [Wed Aug 24 11:48:49 2022] PUT /identity/v3/projects/71a2d9f0a96b4c8b82f343ecc8cc26ba/users/c7974c9be50c4a05b8d067fa0e4d0a96/roles/bbd3afeaf4a9469aa9d6cbf7a5cb9f18 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:48:50.001328 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a8b4a-4bd2-4626-acf0-c7da3ebe5c80 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:50.001611 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a8b4a-4bd2-4626-acf0-c7da3ebe5c80 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:50.001878 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a8b4a-4bd2-4626-acf0-c7da3ebe5c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:50.035180 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-f39a8b4a-4bd2-4626-acf0-c7da3ebe5c80 None None] MFA Rules not processed for user `c7974c9be50c4a05b8d067fa0e4d0a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:50.038497 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f39a8b4a-4bd2-4626-acf0-c7da3ebe5c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:50.073508 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:50.082256 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7609f88d3e834d29a09d17d3a0ba8614', 'request_id': 'req-8bff07e8-8244-4209-a617-8bf0d77ecb83'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:50.088596 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7609f88d3e834d29a09d17d3a0ba8614', 'request_id': 'req-8bff07e8-8244-4209-a617-8bf0d77ecb83'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:50.137349 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1465/2878] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 11:48:50.153936 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a8b4a-4bd2-4626-acf0-c7da3ebe5c80 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:50.155686 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-f39a8b4a-4bd2-4626-acf0-c7da3ebe5c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:50.177033 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7609f88d3e834d29a09d17d3a0ba8614', 'request_id': 'req-8bff07e8-8244-4209-a617-8bf0d77ecb83'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:50.208333 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] The token cache is being invalidate because project 7609f88d3e834d29a09d17d3a0ba8614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:48:50.208845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:50.209691 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-8bff07e8-8244-4209-a617-8bf0d77ecb83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:48:50.224957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1414/2879] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:48:49 2022] DELETE /identity/v3/projects/7609f88d3e834d29a09d17d3a0ba8614 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 11:48:50.269803 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a8b4a-4bd2-4626-acf0-c7da3ebe5c80 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:50.390839 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5ce16d5ea8240debfcb69ac7b1f71c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:50.391895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:50.391895 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:50.392133 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:50.396350 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:50.547795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:50.549323 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:50.570029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1466/2880] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:48:51.085849 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:51.837215 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:52.624970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fd50d-0cb9-4f09-ac00-895dcd837461 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:52.624970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fd50d-0cb9-4f09-ac00-895dcd837461 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:52.624970 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fd50d-0cb9-4f09-ac00-895dcd837461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:52.662822 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-696fd50d-0cb9-4f09-ac00-895dcd837461 None None] MFA Rules not processed for user `14a74be492a5403c8c9750bb6d6c4c33`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:52.664309 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-696fd50d-0cb9-4f09-ac00-895dcd837461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:52.752454 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1415/2881] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:52.766248 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:52.767731 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:52.848237 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeac9aa-96d7-49b1-8c6b-4dc4e23faa30 service cinder] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:52.851945 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75ac781cf2d84d95a789724e74e12acf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:52.852564 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:52.852985 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:52.853200 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:52.857318 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:52.891165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=100 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:48:52.933368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:52.934938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:52.946113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1467/2882] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:48:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 11:48:53.190594 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1a839-fc91-4ff7-ab3e-a6ebd18e2818 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:53.191147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1a839-fc91-4ff7-ab3e-a6ebd18e2818 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:53.191147 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1a839-fc91-4ff7-ab3e-a6ebd18e2818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:53.215580 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-69f1a839-fc91-4ff7-ab3e-a6ebd18e2818 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:53.216971 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-69f1a839-fc91-4ff7-ab3e-a6ebd18e2818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:53.283845 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1416/2883] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:48:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:48:53.462305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:48:53.463843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:53.546261 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec009861-4a88-41f1-8a66-77e30ce9c7c7 service glance] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:48:53.552755 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184f9bd-d0f0-4a84-9670-0c12742cfdfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:48:53.553275 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184f9bd-d0f0-4a84-9670-0c12742cfdfe service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:53.553511 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184f9bd-d0f0-4a84-9670-0c12742cfdfe service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:53.553777 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184f9bd-d0f0-4a84-9670-0c12742cfdfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:53.567782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184f9bd-d0f0-4a84-9670-0c12742cfdfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:48:53.569413 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184f9bd-d0f0-4a84-9670-0c12742cfdfe service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:48:53.578765 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1468/2884] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:48:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:48:53.586472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e011-0c28-413d-9c63-eca08596fca8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:48:53.586472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e011-0c28-413d-9c63-eca08596fca8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:48:53.586472 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e011-0c28-413d-9c63-eca08596fca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:48:53.605237 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-85b4e011-0c28-413d-9c63-eca08596fca8 None None] MFA Rules not processed for user `c7974c9be50c4a05b8d067fa0e4d0a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:48:53.606527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-85b4e011-0c28-413d-9c63-eca08596fca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:48:53.618094 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1417/2885] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:48:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:03.574932 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=47 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:03.599346 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=63 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:05.682856 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704d476-9f7e-4c7d-9881-663c1d9d0292 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:05.684202 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704d476-9f7e-4c7d-9881-663c1d9d0292 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:05.684585 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704d476-9f7e-4c7d-9881-663c1d9d0292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:05.706848 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-3704d476-9f7e-4c7d-9881-663c1d9d0292 None None] MFA Rules not processed for user `5591708bfed04ac185d4c610e8b76125`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:05.709161 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-3704d476-9f7e-4c7d-9881-663c1d9d0292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:05.717203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1469/2886] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:49:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:07.608413 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a87b7-2d61-4b8e-b7e3-e3e08693fcb8 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:07.608897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a87b7-2d61-4b8e-b7e3-e3e08693fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:07.608998 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a87b7-2d61-4b8e-b7e3-e3e08693fcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:07.629299 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-b96a87b7-2d61-4b8e-b7e3-e3e08693fcb8 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:07.630523 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-b96a87b7-2d61-4b8e-b7e3-e3e08693fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:07.639557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1418/2887] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:49:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:07.655737 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704d476-9f7e-4c7d-9881-663c1d9d0292 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:07.668500 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704d476-9f7e-4c7d-9881-663c1d9d0292 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:07.678770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa7abd-068d-47b6-a597-c117da1340e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:07.679096 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa7abd-068d-47b6-a597-c117da1340e0 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:07.679371 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa7abd-068d-47b6-a597-c117da1340e0 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:07.679640 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa7abd-068d-47b6-a597-c117da1340e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:07.689990 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa7abd-068d-47b6-a597-c117da1340e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:07.691827 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa7abd-068d-47b6-a597-c117da1340e0 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:07.701149 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1470/2888] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:49:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:49:10.699014 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d1a4f-1fb6-42b0-bde1-de6f2fba8383 None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:10.699612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d1a4f-1fb6-42b0-bde1-de6f2fba8383 None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:10.699612 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d1a4f-1fb6-42b0-bde1-de6f2fba8383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:10.726388 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-ce4d1a4f-1fb6-42b0-bde1-de6f2fba8383 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:10.727696 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-ce4d1a4f-1fb6-42b0-bde1-de6f2fba8383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:10.740963 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1419/2889] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:49:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:10.750769 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa7abd-068d-47b6-a597-c117da1340e0 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:10.762770 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa7abd-068d-47b6-a597-c117da1340e0 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:10.773359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e6437-d74f-47a7-ad6d-a12c23ed6cf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:10.773986 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e6437-d74f-47a7-ad6d-a12c23ed6cf2 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:10.774333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e6437-d74f-47a7-ad6d-a12c23ed6cf2 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:10.774608 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e6437-d74f-47a7-ad6d-a12c23ed6cf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:10.785173 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e6437-d74f-47a7-ad6d-a12c23ed6cf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:10.786830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e6437-d74f-47a7-ad6d-a12c23ed6cf2 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:10.799208 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1471/2890] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:49:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:49:20.718355 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=6 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:20.793880 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=15 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:29.599872 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d97e6d-f834-48e0-bffb-8ff3dd7fce3b None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:29.600507 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d97e6d-f834-48e0-bffb-8ff3dd7fce3b None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:29.600507 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d97e6d-f834-48e0-bffb-8ff3dd7fce3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:29.623554 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-78d97e6d-f834-48e0-bffb-8ff3dd7fce3b None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:29.625224 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-78d97e6d-f834-48e0-bffb-8ff3dd7fce3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:29.635134 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1420/2891] 10.222.0.84 () {58 vars in 967 bytes} [Wed Aug 24 11:49:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:29.652158 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e6437-d74f-47a7-ad6d-a12c23ed6cf2 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:29.668165 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e6437-d74f-47a7-ad6d-a12c23ed6cf2 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:29.675645 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdf420-b1e6-4bd9-9b87-5515dc61e9c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:29.683570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdf420-b1e6-4bd9-9b87-5515dc61e9c8 service swift] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:29.683570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdf420-b1e6-4bd9-9b87-5515dc61e9c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:29.683570 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdf420-b1e6-4bd9-9b87-5515dc61e9c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:29.687910 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdf420-b1e6-4bd9-9b87-5515dc61e9c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:29.690314 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdf420-b1e6-4bd9-9b87-5515dc61e9c8 service swift] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:29.703869 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1472/2892] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:49:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:49:29.986407 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d97e6d-f834-48e0-bffb-8ff3dd7fce3b None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:29.988000 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-78d97e6d-f834-48e0-bffb-8ff3dd7fce3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:30.071715 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d97e6d-f834-48e0-bffb-8ff3dd7fce3b None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:30.078133 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4b297-3fad-41bb-b912-ba7de7662d7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f58a559951c94d68951da78c82341a00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:30.082896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4b297-3fad-41bb-b912-ba7de7662d7d service nova] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:30.082896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4b297-3fad-41bb-b912-ba7de7662d7d service nova] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:30.082896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4b297-3fad-41bb-b912-ba7de7662d7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:30.082896 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-c5c4b297-3fad-41bb-b912-ba7de7662d7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:30.096087 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4b297-3fad-41bb-b912-ba7de7662d7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:30.097735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4b297-3fad-41bb-b912-ba7de7662d7d service nova] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:30.110343 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1421/2893] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:49:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:49:33.083947 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdf420-b1e6-4bd9-9b87-5515dc61e9c8 service swift] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:33.085641 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-6cbdf420-b1e6-4bd9-9b87-5515dc61e9c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:33.158410 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdf420-b1e6-4bd9-9b87-5515dc61e9c8 service swift] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:33.162960 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:33.163581 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:33.163859 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:33.164129 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] PATH_INFO: `/v3/users/3afc9f4d5bc3473c837926583d334d31` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:33.165707 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3afc9f4d5bc3473c837926583d334d31)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:33.167146 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:33.341176 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3afc9f4d5bc3473c837926583d334d31'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.401562 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3afc9f4d5bc3473c837926583d334d31'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.415835 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3afc9f4d5bc3473c837926583d334d31'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.416513 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3afc9f4d5bc3473c837926583d334d31'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.424549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-3f06c721-1b4e-4816-9bce-fb270fde7b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3afc9f4d5bc3473c837926583d334d31'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.430721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1473/2894] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:49:33 2022] DELETE /identity/v3/users/3afc9f4d5bc3473c837926583d334d31 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:49:33.680610 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4b297-3fad-41bb-b912-ba7de7662d7d service nova] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:33.691662 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4b297-3fad-41bb-b912-ba7de7662d7d service nova] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:33.750616 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:33.751277 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:33.751551 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:33.751819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] PATH_INFO: `/v3/projects/1d19d73eda154313be8539147f42ec11` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:33.752959 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d19d73eda154313be8539147f42ec11)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:33.755140 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:33.852889 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d19d73eda154313be8539147f42ec11', 'request_id': 'req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.858359 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d19d73eda154313be8539147f42ec11', 'request_id': 'req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.953383 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d19d73eda154313be8539147f42ec11', 'request_id': 'req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.964557 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] The token cache is being invalidate because project 1d19d73eda154313be8539147f42ec11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:49:33.964871 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.965493 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:33.966983 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1422/2895] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:49:33 2022] DELETE /identity/v3/projects/1d19d73eda154313be8539147f42ec11 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:49:34.560298 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05f922-b358-4e61-a8f0-e56cd490edb8 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:34.561590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05f922-b358-4e61-a8f0-e56cd490edb8 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:34.561590 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05f922-b358-4e61-a8f0-e56cd490edb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:34.581219 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-cf05f922-b358-4e61-a8f0-e56cd490edb8 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:34.582469 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-cf05f922-b358-4e61-a8f0-e56cd490edb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:34.654626 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1474/2896] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:49:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:34.667183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:34.668583 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:34.680649 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:34.743670 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1ff7f-6930-4391-9cb4-a07c4908f77a admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:34.749611 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a11f7d-2abe-415c-be00-ffd9e33a6473 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:34.750276 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11f7d-2abe-415c-be00-ffd9e33a6473 service swift] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:34.750600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11f7d-2abe-415c-be00-ffd9e33a6473 service swift] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:34.750870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a11f7d-2abe-415c-be00-ffd9e33a6473 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:34.758988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a11f7d-2abe-415c-be00-ffd9e33a6473 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:34.760537 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a11f7d-2abe-415c-be00-ffd9e33a6473 service swift] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:34.768527 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1423/2897] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:49:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:49:35.759223 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f357c-64fd-4f52-a9a7-f89f27f51855 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:35.759724 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f357c-64fd-4f52-a9a7-f89f27f51855 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:35.759838 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f357c-64fd-4f52-a9a7-f89f27f51855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:35.780128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-852f357c-64fd-4f52-a9a7-f89f27f51855 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:35.781477 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-852f357c-64fd-4f52-a9a7-f89f27f51855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:35.788677 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1475/2898] 10.222.0.84 () {58 vars in 967 bytes} [Wed Aug 24 11:49:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:35.803438 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a11f7d-2abe-415c-be00-ffd9e33a6473 service swift] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:35.812421 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a11f7d-2abe-415c-be00-ffd9e33a6473 service swift] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:35.817313 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadbbb2-a206-4869-b0c2-470f730be640 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:35.817897 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadbbb2-a206-4869-b0c2-470f730be640 service swift] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:35.818159 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadbbb2-a206-4869-b0c2-470f730be640 service swift] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:35.818419 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadbbb2-a206-4869-b0c2-470f730be640 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:35.826317 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadbbb2-a206-4869-b0c2-470f730be640 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:35.828499 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aadbbb2-a206-4869-b0c2-470f730be640 service swift] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:35.836325 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1424/2899] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:49:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:49:36.510049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c72a9-065b-437e-946e-d4bb568bd027 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:36.510351 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c72a9-065b-437e-946e-d4bb568bd027 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:36.510653 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c72a9-065b-437e-946e-d4bb568bd027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:36.519700 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:36.531672 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-de8c72a9-065b-437e-946e-d4bb568bd027 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:36.532921 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-de8c72a9-065b-437e-946e-d4bb568bd027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:36.540067 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1476/2900] 10.222.0.84 () {58 vars in 1009 bytes} [Wed Aug 24 11:49:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:36.552513 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadbbb2-a206-4869-b0c2-470f730be640 service swift] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:36.563100 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadbbb2-a206-4869-b0c2-470f730be640 service swift] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:36.568663 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b0450-6fed-4c91-98c3-34155ae626b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:36.569285 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b0450-6fed-4c91-98c3-34155ae626b9 service swift] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:36.569559 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b0450-6fed-4c91-98c3-34155ae626b9 service swift] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:36.569824 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b0450-6fed-4c91-98c3-34155ae626b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:36.578327 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b0450-6fed-4c91-98c3-34155ae626b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:36.583240 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b0450-6fed-4c91-98c3-34155ae626b9 service swift] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:36.587916 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1425/2901] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:49:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:49:37.201333 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9de8e-973e-48ba-a477-911e04af417e None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:37.202126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9de8e-973e-48ba-a477-911e04af417e None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:37.202126 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9de8e-973e-48ba-a477-911e04af417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:37.226398 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-aed9de8e-973e-48ba-a477-911e04af417e None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:37.229305 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aed9de8e-973e-48ba-a477-911e04af417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:37.236292 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1477/2902] 10.222.0.84 () {58 vars in 967 bytes} [Wed Aug 24 11:49:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:37.249618 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b0450-6fed-4c91-98c3-34155ae626b9 service swift] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:37.260564 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b0450-6fed-4c91-98c3-34155ae626b9 service swift] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:37.265775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ab88e-80c4-4f98-8e71-263d25fb1fa3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:37.266369 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ab88e-80c4-4f98-8e71-263d25fb1fa3 service swift] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:37.266692 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ab88e-80c4-4f98-8e71-263d25fb1fa3 service swift] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:37.266956 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ab88e-80c4-4f98-8e71-263d25fb1fa3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:37.275269 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ab88e-80c4-4f98-8e71-263d25fb1fa3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:37.276852 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ab88e-80c4-4f98-8e71-263d25fb1fa3 service swift] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:37.285558 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1426/2903] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:49:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:49:40.140766 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9de8e-973e-48ba-a477-911e04af417e None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:40.142247 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-aed9de8e-973e-48ba-a477-911e04af417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:40.208203 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9de8e-973e-48ba-a477-911e04af417e None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:40.212623 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:40.213291 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:40.213559 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:40.213825 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] PATH_INFO: `/v3/users/c7974c9be50c4a05b8d067fa0e4d0a96` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:40.214938 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7974c9be50c4a05b8d067fa0e4d0a96)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:40.216847 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:40.435599 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7974c9be50c4a05b8d067fa0e4d0a96'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:40.509843 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7974c9be50c4a05b8d067fa0e4d0a96'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:40.522461 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7974c9be50c4a05b8d067fa0e4d0a96'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:40.523111 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7974c9be50c4a05b8d067fa0e4d0a96'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:40.530289 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7974c9be50c4a05b8d067fa0e4d0a96'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:40.536368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1478/2904] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:49:40 2022] DELETE /identity/v3/users/c7974c9be50c4a05b8d067fa0e4d0a96 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:49:40.541186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e08f69-8ccd-45a8-a46f-0a670cb868db None None] REQUEST_METHOD: `POST` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:40.541186 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e08f69-8ccd-45a8-a46f-0a670cb868db None None] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:40.541756 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e08f69-8ccd-45a8-a46f-0a670cb868db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:40.562769 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.auth.core [None req-66e08f69-8ccd-45a8-a46f-0a670cb868db None None] MFA Rules not processed for user `4fac3b46e24449e193279323e9d64fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81707) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:49:40.564302 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-66e08f69-8ccd-45a8-a46f-0a670cb868db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:40.628532 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1427/2905] 10.222.0.84 () {58 vars in 958 bytes} [Wed Aug 24 11:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:49:40.638372 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:40.640016 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:40.705563 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57dd7c-a488-44cb-835c-4e27f42698f9 admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:40.709205 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfd2cc-9509-4ef7-8e12-363ba97c4fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:40.709830 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfd2cc-9509-4ef7-8e12-363ba97c4fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:40.710113 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfd2cc-9509-4ef7-8e12-363ba97c4fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:40.710387 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfd2cc-9509-4ef7-8e12-363ba97c4fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:40.720175 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdfd2cc-9509-4ef7-8e12-363ba97c4fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:40.721648 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdfd2cc-9509-4ef7-8e12-363ba97c4fe2 service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:40.731162 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1479/2906] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:49:40.977633 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e08f69-8ccd-45a8-a46f-0a670cb868db None None] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:40.979155 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-66e08f69-8ccd-45a8-a46f-0a670cb868db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:40.993221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e08f69-8ccd-45a8-a46f-0a670cb868db None None] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:40.998126 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:40.999373 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:40.999688 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:41.000013 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] PATH_INFO: `/v3/projects/71a2d9f0a96b4c8b82f343ecc8cc26ba` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:41.001410 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a2d9f0a96b4c8b82f343ecc8cc26ba)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:41.003664 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:41.128600 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a2d9f0a96b4c8b82f343ecc8cc26ba', 'request_id': 'req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:41.132678 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:41.134920 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a2d9f0a96b4c8b82f343ecc8cc26ba', 'request_id': 'req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:41.243771 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a2d9f0a96b4c8b82f343ecc8cc26ba', 'request_id': 'req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:41.257859 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] The token cache is being invalidate because project 71a2d9f0a96b4c8b82f343ecc8cc26ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81707) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:49:41.258213 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:41.258878 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:49:41.260761 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1428/2907] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:49:40 2022] DELETE /identity/v3/projects/71a2d9f0a96b4c8b82f343ecc8cc26ba => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:49:50.726892 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:51.273629 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:DELETE=2,keystone:SELECT=5,keystone:INSERT=1 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:49:52.637049 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfd2cc-9509-4ef7-8e12-363ba97c4fe2 service neutron] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:49:52.647059 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfd2cc-9509-4ef7-8e12-363ba97c4fe2 service neutron] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:49:52.698556 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a31ad-d9bd-43bd-8b56-aad3acf35ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '372e96bd3361434e9169b783f0c9ea13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:49:52.699186 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a31ad-d9bd-43bd-8b56-aad3acf35ccf service neutron] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:49:52.699463 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a31ad-d9bd-43bd-8b56-aad3acf35ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:49:52.699733 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a31ad-d9bd-43bd-8b56-aad3acf35ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:49:52.701529 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-209a31ad-d9bd-43bd-8b56-aad3acf35ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:49:52.775507 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a31ad-d9bd-43bd-8b56-aad3acf35ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:49:52.776989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a31ad-d9bd-43bd-8b56-aad3acf35ccf service neutron] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:49:52.791812 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1480/2908] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:49:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:50:02.782359 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=50 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:50:48.292911 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:50:48.294365 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:50:48.444775 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c8c5f-ab90-48f3-94bb-7a0328eac7a5 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:50:48.449817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:50:48.453857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:50:48.453857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:50:48.453857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:50:48.453857 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:50:48.465253 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:50:48.466898 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:50:48.477571 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1429/2909] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:50:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:50:58.473081 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=63 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:52:26.572922 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b604cf8-4ac1-41b3-9cda-d1f88fc95c51 None None] REQUEST_METHOD: `POST` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:26.573632 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b604cf8-4ac1-41b3-9cda-d1f88fc95c51 None None] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:26.573905 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b604cf8-4ac1-41b3-9cda-d1f88fc95c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:26.593989 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.auth.core [None req-5b604cf8-4ac1-41b3-9cda-d1f88fc95c51 None None] MFA Rules not processed for user `164f3bcc6363445da191bd1b78499a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81706) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 11:52:26.595172 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5b604cf8-4ac1-41b3-9cda-d1f88fc95c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:26.663799 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1481/2910] 10.222.0.84 () {58 vars in 967 bytes} [Wed Aug 24 11:52:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 11:52:26.796028 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:26.797461 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:26.865869 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0f6fa-8b14-4e70-beec-8f9eaf486a71 service placement] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:26.870988 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94c2c-5089-4115-b3fa-cea89ae315c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ba54c2312b348b08e0faae745747e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:26.871585 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94c2c-5089-4115-b3fa-cea89ae315c3 service swift] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:26.871870 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94c2c-5089-4115-b3fa-cea89ae315c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:26.872143 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94c2c-5089-4115-b3fa-cea89ae315c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:26.880129 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d94c2c-5089-4115-b3fa-cea89ae315c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:26.882195 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d94c2c-5089-4115-b3fa-cea89ae315c3 service swift] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:26.890221 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1430/2911] 10.222.0.84 () {60 vars in 1384 bytes} [Wed Aug 24 11:52:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 11:52:32.078920 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b604cf8-4ac1-41b3-9cda-d1f88fc95c51 None None] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:32.083132 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-5b604cf8-4ac1-41b3-9cda-d1f88fc95c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:32.096549 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b604cf8-4ac1-41b3-9cda-d1f88fc95c51 None None] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:32.102518 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc0aa4-acef-49b7-acf6-5c9da4447f9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:32.103194 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc0aa4-acef-49b7-acf6-5c9da4447f9e service placement] REQUEST_METHOD: `GET` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:32.103534 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc0aa4-acef-49b7-acf6-5c9da4447f9e service placement] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:32.103894 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc0aa4-acef-49b7-acf6-5c9da4447f9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:32.105842 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-d4cc0aa4-acef-49b7-acf6-5c9da4447f9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:32.118713 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc0aa4-acef-49b7-acf6-5c9da4447f9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:32.120304 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cc0aa4-acef-49b7-acf6-5c9da4447f9e service placement] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:32.131142 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1482/2912] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:52:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:52:35.892206 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94c2c-5089-4115-b3fa-cea89ae315c3 service swift] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:35.896476 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-d2d94c2c-5089-4115-b3fa-cea89ae315c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:36.008892 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94c2c-5089-4115-b3fa-cea89ae315c3 service swift] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:36.013957 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:36.014601 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:36.014886 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:36.015160 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] PATH_INFO: `/v3/users/5591708bfed04ac185d4c610e8b76125` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:36.016303 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5591708bfed04ac185d4c610e8b76125)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:36.018270 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:36.172819 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5591708bfed04ac185d4c610e8b76125'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.173399 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5591708bfed04ac185d4c610e8b76125'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.174025 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5591708bfed04ac185d4c610e8b76125'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.181331 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5591708bfed04ac185d4c610e8b76125'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.186447 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:52:36.193223 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5591708bfed04ac185d4c610e8b76125'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.265735 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1431/2913] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:52:35 2022] DELETE /identity/v3/users/5591708bfed04ac185d4c610e8b76125 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:52:36.502833 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc0aa4-acef-49b7-acf6-5c9da4447f9e service placement] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:36.514681 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc0aa4-acef-49b7-acf6-5c9da4447f9e service placement] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:36.581775 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:36.582399 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:36.582676 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:36.582948 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] PATH_INFO: `/v3/projects/9a70cdccbcb84216ae61c72d6736bd63` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:36.584643 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a70cdccbcb84216ae61c72d6736bd63)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:36.587029 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:36.724574 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a70cdccbcb84216ae61c72d6736bd63', 'request_id': 'req-20666ed1-c454-476a-b3ed-83bc4d9c16ca'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.732368 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a70cdccbcb84216ae61c72d6736bd63', 'request_id': 'req-20666ed1-c454-476a-b3ed-83bc4d9c16ca'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.808278 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a70cdccbcb84216ae61c72d6736bd63', 'request_id': 'req-20666ed1-c454-476a-b3ed-83bc4d9c16ca'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.820110 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] The token cache is being invalidate because project 9a70cdccbcb84216ae61c72d6736bd63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:52:36.820402 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.821116 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:36.828634 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1483/2914] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:52:36 2022] DELETE /identity/v3/projects/9a70cdccbcb84216ae61c72d6736bd63 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:52:36.828860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:36.828860 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:36.842745 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e5bed-82c7-4948-9964-7d09d9159464 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:36.898116 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:36.898805 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:36.898995 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:36.899262 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] PATH_INFO: `/v3/users/14a74be492a5403c8c9750bb6d6c4c33` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:36.900449 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a74be492a5403c8c9750bb6d6c4c33)` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:36.902371 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:37.125650 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a74be492a5403c8c9750bb6d6c4c33'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.126414 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a74be492a5403c8c9750bb6d6c4c33'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.127020 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a74be492a5403c8c9750bb6d6c4c33'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.134750 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a74be492a5403c8c9750bb6d6c4c33'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.146185 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.notifications [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a74be492a5403c8c9750bb6d6c4c33'} {{(pid=81707) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.237422 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1432/2915] 10.222.0.84 () {58 vars in 1224 bytes} [Wed Aug 24 11:52:36 2022] DELETE /identity/v3/users/14a74be492a5403c8c9750bb6d6c4c33 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:52:37.498782 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] Authenticating user token {{(pid=81706) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:52:37.500485 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.fernet_utils [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81706) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:52:37.514167 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20666ed1-c454-476a-b3ed-83bc4d9c16ca admin admin] Validating token access rules against request {{(pid=81706) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:52:37.527654 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:52:37.572816 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fac3b46e24449e193279323e9d64fe0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd4c9f105cf4428d9f0cb67e8376cb7e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81706) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:52:37.573444 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:52:37.573721 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:52:37.573993 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] PATH_INFO: `/v3/projects/e53dcc406e794017a18e3f7b66f5a5cd` {{(pid=81706) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:52:37.575380 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53dcc406e794017a18e3f7b66f5a5cd)` {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:52:37.577944 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] RBAC: Authorization granted {{(pid=81706) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:52:37.672753 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53dcc406e794017a18e3f7b66f5a5cd', 'request_id': 'req-4345623d-ec01-4509-8dca-aa7c2aeaa684'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.681128 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53dcc406e794017a18e3f7b66f5a5cd', 'request_id': 'req-4345623d-ec01-4509-8dca-aa7c2aeaa684'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.770224 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53dcc406e794017a18e3f7b66f5a5cd', 'request_id': 'req-4345623d-ec01-4509-8dca-aa7c2aeaa684'} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.781427 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] The token cache is being invalidate because project e53dcc406e794017a18e3f7b66f5a5cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81706) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 11:52:37.781708 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.782327 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG keystone.notifications [None req-4345623d-ec01-4509-8dca-aa7c2aeaa684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81706) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 11:52:37.783795 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: [pid: 81706|app: 0|req: 1484/2916] 10.222.0.84 () {58 vars in 1233 bytes} [Wed Aug 24 11:52:37 2022] DELETE /identity/v3/projects/e53dcc406e794017a18e3f7b66f5a5cd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 11:52:47.233529 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=59,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:52:47.778277 dv-cinder-854274-2-iscsi devstack@keystone.service[81706]: DEBUG dbcounter [-] [81706] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81706) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 11:55:52.271933 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] Authenticating user token {{(pid=81707) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 11:55:52.274011 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:55:52.350931 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5e6ef-8128-4b67-812e-8d8a6c778435 admin admin] Validating token access rules against request {{(pid=81707) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 11:55:52.414908 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911b7d2-aa5e-4d58-8f7b-978aa55d25ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7cc35f5ddf0446e90ded5c85f21f37d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '93f8726a7b3b483aa8f337bd3466d23f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81707) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 11:55:52.415517 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911b7d2-aa5e-4d58-8f7b-978aa55d25ea service placement] REQUEST_METHOD: `GET` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 11:55:52.415795 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911b7d2-aa5e-4d58-8f7b-978aa55d25ea service placement] SCRIPT_NAME: `/identity` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 11:55:52.416063 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911b7d2-aa5e-4d58-8f7b-978aa55d25ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81707) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 11:55:52.417817 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.fernet_utils [None req-e911b7d2-aa5e-4d58-8f7b-978aa55d25ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81707) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 11:55:52.429056 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911b7d2-aa5e-4d58-8f7b-978aa55d25ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 11:55:52.431183 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911b7d2-aa5e-4d58-8f7b-978aa55d25ea service placement] RBAC: Authorization granted {{(pid=81707) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 11:55:52.440945 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: [pid: 81707|app: 0|req: 1433/2917] 10.222.0.84 () {60 vars in 1365 bytes} [Wed Aug 24 11:55:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 11:56:02.436741 dv-cinder-854274-2-iscsi devstack@keystone.service[81707]: DEBUG dbcounter [-] [81707] Writing DB stats keystone:SELECT=58 {{(pid=81707) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}